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Integrated Academic and Occupational Curricula: identifying Valid indicators 
for Secondary Classrooms Serving Students With and Without Disabilities: 

Final Report 

Recent school-to-work legislation suggests that all students, including those with 
disabilities, should have access to (a) school-based learning that combines high performance 
expectations with applied academics, (b) work-based learning that provides work experience, 
mentoring, and instruction in a broad range of workplace competencies, and (c) school- 
supported connecting activities, such as business partnerships, that bring students together with 
employers and postsecondary education institutions. A key principle of this school-to-work 
model is the integration of academic and occupational learning. 

This report (a) identifies central features of integrated curricula in high school 
classrooms, (b) examines existing research on effects of integrated curricula, (c) summarizes 
problems with assessing implementation of integrated curricula and (d) presents results of an 
exploratory study that demonstrated the construct validity of a set of measures related to 
integrated curricula. Underlying concerns of this inquiry were (a) whether available evidence 
suggested that integrating academic and occupational curricula is a beneficial option for students 
with disabilities and (b) to determine how well features of integrated curricula and instruction fit 
with models of curricula and instruction in special education. A full report is available upon 
request from the first author. The full report contains a detailed literature review and 
methodology section for the validation study, along with sample data collection instruments. 
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Features of Integrated Curricula 



The integration of academic and occupational curricula implies changes and 
enhancements to schools, including: more local control, new resources and departmental 
changes to support collegial work, teacher training, easy access to knowledge resources, 
reconsideration of scheduling and tracking practices, a press for achievement, and sustained 
implementation efforts by key, committed staff (Andrew & Grubb; 1992; Little, 1992; Stasz et 
al., 1992). Most current school reforms call for similar organizational changes, and in this 
regard, integration of curricula is comparable to other secondary education reforms. To 
understand how integrated curricula is similar to, yet distinct from other reforms requires 
examination of proposals for (a) the organization and content of subject matter, and 
(b) classroom teaching and learning activities. 

As envisioned by practitioners and proponents, integrated curricula puts into place a high 
school experience that may be described as functional, challenging, and applied (Lankard, 

1996). Although, as Grubb (1995) acknowledged, each implementer must answer difficult 
questions regarding the degree to which different aspects of the contextual, curricular and 
pedagogical practices will be emphasized, certain central features are evident. First, in an 
integrated curriculum a student will experience vocational content that emphasizes generic and 
general work-related skills and knowledge; vocational experiences are neither omitted nor only 
job-specific. Second, academic and occupational content are blended in a way that draws 
attention to school and work connections; students work with knowledge of both academic 
principles and real world practices. Third, students are more often engaged in applied activities 
that require them to be problem-solvers instead of passive receivers of knowledge. Fourth, 



teachers spend less time as dispensers of knowledge and more time guiding and supporting 
student work. Finally, teachers and students are engaged in cooperative activities. 

Examination and discussion of the constructs comprising integrated 
academic/occupational curricula have occurred primarily in the area of regular education (eg., 
Plihal, J ohnson, Bentley, Morgaine, & Liang, 1992). However, indirect support for some of these 
features does exist in the special education literature. For example, a content blend approach to 
high school curricula would make sense for students with disabilities if that meant providing a 
range of academic, vocational, and other life skills content within an outcome-oriented 
framework that recognized both postsecondary academics and employment as meaningful goals 
for many students with disabilities (Benz & Kocchar, 1996). Also, blending academic and 
occupational courses and programs may facilitate meaningful participation and physical 
placement of students with disabilities in regular education classrooms by making academics 
more available and applied (Phelps, 1992). Further, participation in vocational education has 
been linked to positive employment outcomes for students with disabilities, especially those 
with mild disabilities (Kohler, 1993; Wagner, 1991). However, the goals, curriculum content, 
and instructional methods of the vocational courses that related to positive effects have not been 
systematically assessed. Finally, special education professionals have taken the position that 
special education must be part of secondary education restructuring efforts, such as integrated 
curricula (e g., Benz & Kocchar, 1996), but, there is a need for more information about the 
compatibility of instructional approaches recommended in conjunction with integrated curricula 
and the research-based instructional practices recommended by the special education field. 




4 

5 



Research on Effects of Integrated Curricula 



At this time, the evidence that integrated curricula may benefit students without 
disabilities is indirect; the greatest support comes from evaluations of comprehensive programs, 
such as career academies and magnets. These programs seem to promote student achievement 
and persistence. Empirical evidence of effects for students with disabilities is almost 
nonexistent. Although model integrated programs for students with disabilities exist, limited 
generaliza'ble information on outcomes of these programs is available. 

Previous reviews 

Three recent reviews of the literature on integrated academic/occupational curricula have 
examined practices or related outcomes (Plihal et al., 1992; Stasz, Kaganoff, & Eden, 1994; 
Stem,Finkelstein, Stone, Latting, & Domsife, 1994). Additionally, one reviewer examined 
studies of contextual learning (Karweit, 1994). Two of these reviews were conducted on behalf 
of the most recent National Assessment of Vocational Education (Karweit, 1994; Stasz et al., 
1994). 

Previous reviewers of the literature on integrated academic/occupational curricula have 
pointed out that most studies inadequately defined interventions, and relatively few provided any 
empirical evidence of the effects of integrated curricula. Common methodological problems of 
empirical studies on integration included failure to adequately define program components and 
dependent variables, limited or no information regarding degree or fidelity of implementation, 
infrequent use of comparison groups or statistical controls, and limited reporting of basic 
statistical results. There is some evidence that comprehensive programs such as magnets and 
academies that have an integrated academic/occupational focus may provide academic benefits 
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to some average and low-achieving students considered to be at risk for dropping out (e.g., 

Crain, Heebner, & Si, 1992). However, results are inconsistent across program sites and student 
groups. Studies have not examined postschool outcomes or vocational competencies. To date, 
studies of contextual learning offer only limited and indirect evidence of benefits of such 
teaching practices for secondary students. 

Descriptive studies 

Recent descriptive studies of schools, districts and states have provided additional 
information regarding important features of policy environment, school context, and classroom 
pedagogy that support integration of academic and occupational curricula (Bodilly, Ramsey, 
Stasz, & Eden, 1993; Stasz, et al., 1992). Researchers have identified “exemplary 55 programs for 
students with disabilities that incorporate integrated curricula (Matias, Maddy-Bemstein, & 
Kantenberger, 1995). Additionally, researchers have begun to identify teacher and administrator 
views regarding potential outcomes of integrated curricula (Finch, Schmidt, & Faulkner, 1994) 
or have documented program-reports of general outcomes (Matias et al., 1995). Two studies 
(Matias et al., 1995; Stasz et al., 1992) reported on the participation of students with disabilities 
in programs or classes that featured content or instructional elements of integrated curricula. 
Although each of these studies implied that integrated curricula may promote positive student 
outcomes, none specifically attempted to demonstrate effects. It seems clear that engagement of 
students with disabilities in classes or programs that provide integrated academic/occupational 
curricula is possible, but whether integrated curricula may benefit students with disabilities 
cannot be concluded from these studies. 

Program evaluations 

Recent evaluations of programs that include an integrated curricula component have 
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ranged from studies of large scale model program replications to evaluations conducted by a 
teacher-researcher in a single school. These evaluations have provided only limited evidence to 
support the contention that engagement in integrated curricula leads to benefits for students. The 
principle reason for this situation is that integrated curricula is typically one component among 
many in a complex set of interventions within programs, and these programs may be one of 
many being promoted by reformers, isolating the effects of integrated curricula has been 
difficult also because only a few studies have examined the nature or level of implementation 
(e g.. Stem, Raby, & Dayton, 1992) or have attempted to systematically describe elements of 
implementation (e.g., Penn, 1992; Raber & Merchlinsky, 1995). Further, as found by previous 
reviewers, these evaluation reports — with the exception of the career academy studies (Stem et 
al., 1992) -- generally failed to provide information regarding the types of data analyses used and 
basic statistical results needed to evaluate program effects. Information about the effects for 
students with disabilities is similarly limited. Although at least one (Penn, 1992) included 
students with disabilities and two other programs (Project Coffee, as reported by Leutheuser, 
1994, and the career academies) may have, none of the studies provided sufficient detail to 
determine the types of disability groups that may have been represented. The Southern Regional 
Education Board studies (Bottoms, 1995; Bottoms & Presson, 1995) provided interesting 
baseline data on a large group of students using a national comparison group. Unfortunately, 
without additional detail regarding methodological aspects of the studies, interpretation of future 
evaluation results will be difficult. In general, recent program evaluations have suggested that 
programs with an integrated curricula component may contribute to student persistence, 
academic achievement, and postsecondary engagement. The benefits of integrated curricula 
itself are not clear from these studies. 
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Improving Future Research on effects and the Problem of Assessing Implementation 



Determining effects of integration efforts has not been straightforward for several 
reasons, integrating academic and occupational curricula involves changes in both content and 
pedagogy; multiple classroom features must be defined and measured. Integration may occur 
within or across courses; units of analysis may be students, classes, groups of classes or schools. 
Integration may be coupled with other school organizational changes; effects may be 
confounded by multiple component interventions. Programmatic changes may take several years 
to fully implement and some student outcomes of interest occur post-graduation; studying 
effects may require a longitudinal approach. These issues pose methodological challenges for 
future research on integrated curricula. 

Future research on integration of academic and occupational curricula for students with 
disabilities can strengthen understanding of the processes and effects of integrated curricula by 
attending to and accounting for the following: (a) participation of special populations, including 
those with disabilities, (b) clearly defined multiple measures or indicators of program and 
curriculum components, degree of implementation, and student and organizational outcomes, 

(c) consideration of measures that reflect the underlying values and causal-models of local 
implementers and participants, (d) developmental or longitudinal approach to accommodate or 
control for levels of implementation, and (e) complete reporting of quantitative and qualitative 
results in multiple formats accessible to practitioners, policy makers and researchers. 
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A First Step: A Study to identify Valid Measures of implementation 



initially, a greater understanding of what happens in classes and schools that integrate 
academic and occupational curricula is needed. Few of the existing studies on outcomes of 
integrated curricula also provided information about specific features of implementation. Valid 
indicators of classroom implementation are needed for (a) investigation of classroom factors 
associated with positive student outcomes, and (b) exploration of the validity of current 
conceptual models of curriculum and instruction for secondary students with and without 
disabilities (Kohler, 1993; McDonnell, Burstein, Ormseth, Catterall, & Moody, 1990; Porter, 
1993). 

The primary objective of this study was to validate measures of classroom practice that 
would be sensitive to the level of integration of academic and vocational curricula and that 
might be applied to a variety of secondary courses including those experienced by students with 
disabilities. To further examine validity, secondary objectives of the study were to assess 
(a) reliability of the measures and instruments and (b) relevance of resulting information. If 
validated, the measures and instruments would be useful in future evaluations regarding 
implementation of integrated curricula and research on associated outcomes at the student, 
classroom, school, or district level. 

Methods 

This study used systematic observation methods combined with teacher and 
administrative surveys to examine the construct validity of classroom practice measures related 
to the integration of academic and occupational curricula. 

Measures. The measures incorporated features of curriculum and instruction that, in 



recent school-to-work transition literature (e.g., Adelman, 1989; Grubb, Davis, Lum, Plihal, & 
Morgaine, 1991; Plihal et al., 1991), have been associated with the construct of integrated 
academic/occupational curricula: general vocational content, reinforced/fused content blend, 
coaching instruction strategy, applied student activity, and cooperative instruction grouping. 
Within the context of this study, these measures wee used to demonstrate the construct of 
integrated curricula by differentiating four known groups of classes: (a) academic, (b) 
vocational, (c) integrated academic/occupational, and (d) special education academic. 
Theoretically, classes that integrated academic and occupational curricula would tend to have 
higher ranks on these measures. 

Sample. The sample included 27 classes from two comprehensive high schools that 
participated in the Southern Regional Education Board's High Schools That Work initiative. 
Classes represented academic, vocational, integrated academic/occupational curricula, and 
special education academic. The classes encompassed a variety of subjects (e.g., mathematics, 
cosmetology, applied communications), grades (i.e., 9th through 12th), and academic levels 
(e.g., remedial, honors). 

instruments. The measures of interest were sampled in the four types of classes using 
multiple instruments, instruments included: (a) an Administrative Enrollment form; (b) a Course 
Questionnaire to document teachers' initial estimates of time spent in curriculum and instruction 
areas represented by the measures; (c) Daily Logs, on which, teachers reported daily class 
activities and described each activity using coding categories representing the specific measures 
within each dimension; (d) Observer Logs, which were similar to the Daily Logs, but were 
completed by research staff, and provided additional contextual detail such as total length of 
class and number of students present; and (e) Weekly Logs that provided a vehicle for 



teachers to report anecdotal information. Also, a follow-up survey was used to examine teachers' 
post-study perceptions of the data collection process and initial outcomes of the study. 

Data analysis. The primary focus of data analysis was the construct validity of the 
measures. Construct validity was first examined using a "known group" strategy with the intent 
of demonstrating hypothesized differences between groups (Morris, Fitz-Gibbon, & Lindheim, 
1987). Kruskal-Wallis nonparametric one-way analysis of variance tests were conducted to 
determine if teachers and students in integrated curricula classes spent more time on the 
measures than those in other class types. Construct validity was further examined in a second set 
of analyses using Kendall’s W, a nonparametric test of concordance, to determine the degree to 
which the different instruments yielded similar rankings of the groups on the summary measures. 
Additionally, reliability of the instruments and measures was tested. Specifically, Cohen’s 
kappa, which corrects for chance agreement, was used to determine proportion of agreement on 
coding between teachers and research staff. Finally, the content and social relevance of the 
dimensions and coding categories were explored using descriptive statistics from the follow-up 
survey and anecdotal information compiled from teacher-completed Daily and Weekly Logs, 
incidental conversations with teachers, and informal de-briefing interviews conducted with 
observers. 

Results 

The findings provided support for the construct validity of the proposed measures of 
integrated academic/occupational curricula. Although there was considerable variability among 
individual classes within class types, the measures differentiated between integrated or 
vocational classes and traditional academic or special education academic classes. The measures 
were less sensitive to differences between integrated and vocational classes. These results were 



demonstrated across multiple instruments. 

In this study, integrated academic/occupational classes did in practice provide curricula 
and instruction that were qualitatively different from traditional academic or special education 
academic classes. A clear difference was seen in the amount of time students were engaged in 
hands-on activities and were exposed to both academic and vocational content. Students in 
integrated curricula classes, according to observers, spent on the average half the classtime in 
hands-on activities. Almost a third of the classtime was devoted to activities that reinforced or 
fused academic and vocational content. These applied and blended activities were rarely 
practiced in the traditional academic or special education academic classes. It is interesting to 
note that applied activities and blended curricula also were found more often in traditional 
vocational classes than in traditional or special education academic classes. 

Tests of significance based on mean ranks typically demonstrated that vocational or 
integrated classes could be differentiated from academic or special education classes, except in 
the area of coaching as instruction strategy. Mediating these results is the fact that teacher- 
research assistant interobserver agreement was moderate to low, while interobserver agreement 
among observers was moderate to high. This suggests that the observer logs provided more 
reliable estimates of observable classroom activity. In fact, findings of significant differences 
between class types did vary by instrument type. 

Teachers in integrated curricula classes spent significantly more time covering general 
vocational content than those in academic classes, according to the course questionnaire and 
daily logs. Teachers in integrated and vocational classes spent significantly more time providing 
reinforced/fused content blend than those in academic classes, according to the daily and 
observer logs. There were no significant differences between class types on any instrument for 



Mean Ranks on Summary Variables 



Class Types 



Special 



Summary Variable 
Instrument 


Academic 
n = 6 


Vocational 
n = 7 


integrated 
n = 7 


Education 
n = 7 




General Vocational 
Content 


R 


R 


R 


R 




Course Questionnaire 


6.83 


16.36 


20.07 


11.71 


10.16** 


Daily Logs 


7.50 


15.14 


19.86 


12.57 


8.30* 


Observer Logs 


7.50 


13.71 


16.86 


17.00 


6.90 



Reinforce/>'used 



Content Blend 



Course Questionnaire 


10.75 


16.57 


18.00 


10.21 


5.16 


Daily Logs 


6.00 


19.57 


19.43 


9.86 


14.82* 


Observer Logs 


7.50 


19.14 


17.29 


11.14 


9.05** 



Coaching Instruction 



Strategy 



Course Questionnaire 


9.42 


20.07 


14.43 


11.43 


6.92 


Daily Logs 


11.00 


15.43 


15.50 


13.64 


1.35 


Observer Logs 


14.50 


11.79 


16.57 


13.21 


2.11 



Lab/ Applied Student 



Ac tivit y 



Course Questionnaire 


8.92 


19.50 


15.86 


8.33 


9.86** 


Daily Logs 


5.00 


22.43 


15.57 


11.71 


16.47* 


Observer Logs 


5.50 


19.57 


18.79 


10.93 


13.92** 



Cooperative Instruction 



Grouping 



Course Questionnaire 


10.50 


18.21 


19.43 


7.36 


11.29** 


Daily Logs 


11.75 


15.79 


19.86 


8.29 


8.30* 


Observer Logs 


14.00 


16.07 


16.79 


9.14 


4.17 



a Uncorrected value 
* Significant at p < 0.05 
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Average Percent Time Engaged on Summary Variables 



Class Types 



Summary Variable 
Instrument 


Academic 
n = 6 


Vocational 
n = 7 


Integrated 
n = 7 


Special 
Education 
n = 7 




m (m 


M (SD) 


M (SD) 


M (SD) 


General Vocational 
Content 


Course Questionnaire 


20.33 (16.81) 


57.86 (25.64) 


72.14 (22.52) 


40.71 (32.33) 


Daily Logs 


9.46 (20.14) 


35.44 (35.34) 


60.79 (36.28) 


24.52 (21.66) 


Observer Logs 


0.00 (0.00) 


24.02 (40.62) 


35.58 (34.16) 


20.04 (19.87) 


Reinforced/Fused 
Content Blend 


Course Questionnaire 


37.83 (48.29) 


74.71 (20.26) 


84.14 (11.48) 


35.00 (37.52) 


Daily Logs 


9.46 (20.14) 


73.04 (21.76) 


69.31 (37.15) 


20.27 (20.76) 


Observer Logs 


0.00 (0.00) 


26.54 (16.88) 


31.98 (32.78) 


3.32 (4.39) 


Coaching Instruction 
Strategy 


Course Questionnaire 


16.67(11.69) 


52.86 (26.28) 


30.00 (23.63) 


22.14 (13.80) 


Daily Logs 


19.36 (20.38) 


30.56 (27.88) 


29.37 (15.57) 


27.40 (16.39) 


Observer Logs 


6.88 (11.23) 


2.38 (6.30) 


11.79 (15.59) 


3.81 (6.79) 


Lab/Applied Student 
Activity 


Course Questionnaire 


21.67 (32.35) 


62.86 (18.23) 


52.14 (14.10) 


18.67 (24.55) 


Daily Logs 


11.30 (11.73) 


70.02 (17.03) 


45.13 (25.01) 


29.50 (23.87) 


Observer Logs 


2.50 (6.12) 


55. 17 (22. 82) 


49.44 (28.44) 


21.28 (34.03) 


Cooperati ve Instruction 
Grouping 


Course Questionnaire 


25.00 (15.17) 


50.86 (25:83) 


55:71 (24.91) 


16.71 (8:98) 


Daily Logs 


15.96(15.11) 


30.68 (27.80) 


40.48 (27.28) 


7.84 (7.09) 


Observer Logs 


14.50 (9.37) 


30.71 (31.02) 


30.27 (30.54) 


5.34 (7.85) 
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teachers' use of coaching instruction strategy . According to the observer logs, students in both 
integrated and vocational classes spent significantly more time on lab/applied student activity 
than those in academic classes. The course questionnaire and daily logs also showed this 
difference between vocational and academic classes, as well as between vocational and special 
education classes. Finally, students in integrated classes spent significantly more time on 
cooperative instruction grouping than those in special education classes, according to the course 
questionnaire and daily logs. 

Although findings of significance varied by instrument type, there was a strong 
concordance of rankings among the course questionnaire, daily logs and observer logs on all 
summary variables except coaching. In other words, classes that received higher rankings on one 
instrument also were more likely to receive higher rankings on other instruments. Although 
observers documented less time engaged on summary variables than teachers reported on the 
daily log, and teachers reported less time engaged on summary variables through the daily logs 
than they did on the course questionnaires, a consistent pattern of rankings was evident. 



Kendall's W (Coefficient of Concordance) for Mean Ranks on Instruments 



Summary Variable 


x 2a 


w 


B 


General Vocational Content 


61.58 


0.79 


<0.001 


Reinforced/Fused Content Blend 


50.69 


0.65 


0.003 


Coaching Instruction Strategy 


26.49 


0.34 


0.44 


Lab/Applied Student Activity 


59.75 


0.80 


<0.001 


Cooperative Instruction Grouping 


52.72 


0.68 


0.001 



a df= 26, except Student Activity dimension df= 25 
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Integrated and vocational classes typically received higher ranks and academic and special 
education academic classes typically received lower ranks. 

Teachers generally supported the belief that differences between class types found 
through the daily log represented important differences, suggesting some degree of face validity 
for the measures. Further, teachers expressed support for the idea that collecting classroom 
information about the measures would be useful for a variety of purposes, such as monitoring 
how they use classtime to inform their own professional development. Teachers did not view 
any of the reporting methods as overly intrusive in terms of time required nor disruptive to class 
activities. Some teachers suggested in their written comments on the weekly logs and follow-up 
survey that some class activities may not have been well represented by the daily log data 
because (a) certain activities occur only at certain times in the semester, (b) choosing between 
codes was difficult for some activities, and (c) some classroom activities focused on personal 
life skills, which were difficult to code on academic and vocational dimensions. 

Implications 

This study has two important implications for classroom practice and education policy. 
First, findings suggest that students may experience different types of curriculum and instruction 
in integrated curriculum classes than in other more traditional class types. Students, including 
those with disabilities, who prefer an applied learning approach to academics may find a better 
match in classrooms that provide integrated academic/occupational curricula. Further, it has 
been proposed that a potential benefit of integrated curricula classes is that students would not 
have to choose between academic and applied occupational courses as is often the case in 
current high school settings (Benz & Kochhar, 1996; Phelps, 1992). This study provides some 
preliminary support for this belief. However, this study can not address the question of whether 



the academic content was comparable across class types. It may only be concluded that students 
in this study were more likely to experience both academic and vocational content in integrated 
and vocational classes. 

Second, valid measures of integrated curricula have important uses for program 
evaluation. The measures may be used to monitor implementation of integrated curricula for 
purposes of program development in schools that adopt an integrated curricula approach. The 
measures and instruments in this study would support schools' self-assessments and dialogues 
with parents and other stakeholders regarding the goals and methods used in school programs. 
This type of information would fit into a model-guided method of program implementation 
monitoring (Brekke, 1987). In this type of monitoring, data are collected that reflect the 
underlying purposes or theories of program operation. These data are then used systematically by 
school personnel and stakeholders to inform or refine existing programs. As part of a system of 
school-level indicators these types of measures are a way to purposefully shape improvement 
efforts at multiple system levels (Levesque, Bradby, & Rossi, 1996). 

Also, the measures may be used to assess the relation between implementation of 
integrated curricula components in the classroom and student outcomes such as achievement in 
content areas, school engagement and graduation, and postschool success in employment and 
postsecondary education. Most evaluation studies of programs involving integrated curricula 
have typically ignored levels of implementation in the classroom and have, therefore, failed to 
provide convincing evidence of the impact of integrated curricula. This lack of information 
makes it difficult for practitioners in regular and special education to judge the value of an 
integrated curricula approach for students with and without disabilities. This study begins to 
address this problem by identifying measures that could be used to examine the link between 
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what actually happens in integrated curricula classrooms and student outcomes, such as 
academic achievement. At the school or classroom level, these measures could be used to 
document degree of implementation. This information would provide greater insight into the 
results of evaluations purporting to show different outcomes for different class types. 

Limitations 

This study must be considered an exploratory study. The sample was chosen to permit 
discrimination between class types within a comprehensive school setting. The sample was not 
intended to be representative of other classes or schools. The results related to actual time 
engaged on the summary variables by different class types should not be considered 
representative of academic, vocational, integrated or special education academic classes in other 
schools. The measures were not designed to suggest that spending a specific amount of time on 
particular areas of curriculum and instruction would make a class “integrated.” In fact, there was 
considerable variability among individual classes within class types. Also, the measures did not 
identify significant differences between vocational and integrated classes. This may be a result 
of limited implementation in the sample schools, poor reliability, or that the five dimensions and 
their categories were not sufficiently defined to differentiate between the two class types. It is 
important to note that in situations where teachers might be held accountable for their 
performance on the measures, the data would be subject to further bias and would be considered 
less reliable than in this study. 

A further limitation of the study is that, although the measures were based on the 
available literature, the definitions used may not address areas of concern in other schools. For 
example, there was no attempt in this study to examine the depth or breadth of academic topics 
available in the classes. Also, the special education students in this study participated in self- 
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contained academic classes or were mainstreamed in regular education classes. They were 
typically students with mild to moderate mental retardation or learning disabilities. If classes 
serving students with other types of disabilities (e.g., behavior disorders) or classes arranged in 
other ways (e.g., special education self-contained vocational) had been included in the study, 
participants might have identified other dimensions of curriculum and instruction that should be 
considered as relating to integrated curricula. Similarly, another limitation of this study is that 
only school-based classrooms were examined. Classes that were based in the community or 
work settings may have important features that were not considered in relation to the measures 
tested in this study. 

Future Research on Measures and Implementation of Integrated Curricula 

Based on these implications and limitations, four areas for future research on integrated 
curricula are suggested. 

Further examine validity and reliability. First, the validity of the measures should be 
further assessed in other schools, in both school- and community-based classes, and with other 
types of students with disabilities. Second, additional measures should be included that would 
help to extend and refine the definition of integrated curricula. These additional measures should 
systematically document: (a) blending of other functional contexts (e.g., life skills) with 
academics, and (b) academic content and skill level of activities. Seeking expert consensus on 
these measures would further enhance their validity. Finally, variability within class types might 
be reduced by creating larger samples from similar academic content areas (e.g., mathematics). 

Program evaluation. A fundamental issue is to determine to what degree components of 
integrated curricula must be present to create meaningful differences in student outcomes and 
the types of outcomes that should be documented. Second, the degree to which teachers must 



make the connections between academic and vocational content areas explicit for students in 
order to demonstrate relevance is not clear. Third, features of integrated curricula as they exist in 
context at different schools and programs must be documented. Also, it may be useful to further 
develop the course questionnaire to provide an efficient and accurate representation of 
classroom activities. 

Students with disabilities. First, researchers must consider that lifeskills as well as 
vocational content may provide students with and without disabilities alternate and equally 
productive contexts for learning academic material. Second, the concept of high academic 
expectations for secondary students with cognitive disabilities must be examined, especially in 
regard to adequately defining integrated curricula. Third, the contention that a general vocational 
content approach is best must be examined in light of the special education literature support for 
students with severe disabilities to have specific skill training accompanied by generalization to 
multiple environments. Finally, researchers should investigate the supports needed and delivered 
to facilitate participation of students with disabilities in integrated curricula classes. 

Personnel preparation. In this study, teachers from academic backgrounds were able to 
implement curricula and instruction such that their classes looked similar to vocational classes 
on the measures of interest. This finding raises questions regarding the type of personnel 
preparation needed to successfully implement integrated curricula. 

Conclusion 

This study explored in a systematic manner important aspects of curriculum and 
instruction in secondary classrooms and how these relate to the construct of integrated 
academic/occupational curricula found in the literature. As a result of this study, (a) the 
construct of integrated curricula was shown to be observable and measurable, (b) features of 
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current conceptual models of integrated curricula were shown to have validity in a variety of 
classrooms, (c) important limitations of the integrated curricula construct were identified, (d) the 
reliability of teacher reports was assessed, and (e) teachers' assessments of the relevance of the 
resulting information were explored. 

Such implementation information is needed to determine if integrated curricula, a major 
component of current school-to-work transition initiatives, is, in fact, beneficial for all students. 
The School-to-Work Opportunities Act, the Perkins Act Amendments and Individuals with 
Disabilities Education Act Amendments of 1990 have created an opportunity to put in place a 
comprehensive school-to-work system that creates high academic expectations and promotes 
positive student outcomes for all students. To achieve this goal, more information is needed by 
policymakers and practitioners about classroom-level implementation of school-to-work 
activities and how these activities address the individual educational needs of students with 
disabilities. This study addressed this need in two important ways: (a) completion of a 
methodological component of indicator development that is generally not possible for school 
practitioners and administrators to undertake, and (b) demonstration of the feasibility of using 
observable classroom practice measures to describe integration of academic and occupational 
curricula. 
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ABSTRACT 



The purpose of this study was to explore the validity of a set of observable measures 
representing integrated academic/occupational curricula in high school classrooms. These 
measures incorporated features of curriculum and instruction that, in recent school-to-work 
transition literature, have been associated with the construct of integrated academic/occupational 
curricula: general vocational content, reinforced/fused content blend, coaching instruction 
strategy, applied student activity, and cooperative instruction grouping. The sample included 27 
classes from two comprehensive high schools. Classes represented academic, vocational, 
integrated academic/occupational curricula, and special education academic. Teachers 
completed a course questionnaire and daily logs that described classroom activities using the 
measures. Research staff conducted classroom observations. 

Of particular interest was whether the measures would reflect significant differences 
between class types and whether these differences could be observed across multiple 
instruments. Theoretically, integrated curricula classes would have higher mean ranks on the 
measures than would other class types. Data analyses included non-parametric tests for analyses 
of variance and concordance. Kappa tests were used to analyze interobserver agreement. The 
content and social relevance of the measures were explored using descriptive statistics from a 
follow-up survey and anecdotal information. 

All instruments showed that, on the average, teachers and students in integrated classes 
spent more time engaged in activities represented by the measures than those in academic or 
special education academic classes. Teachers and students in vocational classes also spent more 
time on these variables, except for coaching as reported by observer logs, than their counterparts 



in academic and special education classes. Tests of significance based on mean ranks typically 
demonstrated that integrated curricula or vocational classes could be differentiated from 
academic or special education classes, except for coaching as instruction strategy. Teacher- 
research staff interobserver agreement was moderate to low; interobserver agreement among 
observers was moderate to high. Teachers reported that the measures and instruments were easy 
to use and produced useful descriptive information. The discussion addresses limitations of the 
measures and the study, implications for classroom and school practice, and suggestions for 
future research. 
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CHAPTER I 



INTRODUCTION 
School-to-Work for All Students 

The transition from school to postsecondary adult life is critical for adolescents with and 
without disabilities (Benz & Kochhar, 1996; Halpem, 1994). During this time, students make 
decisions regarding school completion, postsecondary education, career interests, and 
employment options. Ideally, secondary education experiences will provide supportive 
opportunities for students to engage in high quality instructional programs relevant to 
postsecondary options, explore their interests and preferences, engage in community-based 
learning activities, and establish connections with businesses, postsecondary institutions, and 
community agencies (Benz & Kochhar, 1996; Halpem, 1994; Kazis, 1993; Kohler, 1993; 
Winters, 1993). Current federal legislation affecting regular and special education emphasizes 
the importance of school-to-work transition activities to promote positive adult outcomes for 
students with and without disabilities (Individuals with Disabilities Education Act [IDEA], 1992; 
School-to-Work Opportunities Act [STWOA], 1994). 

According to the School-to-Work legislation (1994) all students will benefit from 
exposure to (a) school-based learning that combines high performance expectations with applied 
academics, (b) work-based learning that provides work experience, mentoring, and instruction in 
a broad range of workplace competencies, and (c) school-supported connecting activities, such 
as business partnerships, that bring students together with employers and postsecondary 
education institutions. 

In regard to this model, there is discussion in both the regular and special education 
fields regarding the nature of secondary curriculum and instruction within a school-to-work 
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system. The issue ofcurriculum and instruction arises from concerns voiced by both special 
educators and regular educators that some students (e.g., students with and without disabilities 
who participate in the general track) receive less rigorous and lower-quality instruction (Edgar & 
Polioway, 1994; McDonnell, 1989). There is concern that these students are prepared neither for 
college or work (National Education Goals Panel, 1995). 

This study examines a key principle of the school-to-work model related to curriculum 
and instruction - the integration of academic and occupational learning — as it applies to both 
students with and without disabilities. This study also presents a set of measures and 
instruments for assessing implementation of central features of integrated curricula within a 
broad range of classroom settings. 

Historical and Current Interest in Integrated Curricula 
All students, including those with disabilities, should participate in secondary education 
programs that integrate academic and occupational curricula. This proposition has been a 
central feature of prominent secondary education reforms in the last ten years. The Carl D. 
Perkins Act (1990), the major federal funding vehicle for vocational education in the United 
States, and the School-to-Work Opportunities Act (1994), a key piece of the federal systemic 
education reform agenda, both promote the principle of integrating academic and occupational 
education as a cornerstone of vocational programs and school-to-work systems. Both require 
that students with disabilities have access to these activities. 

The current interest in integrating academic and occupational education can be viewed as 
a resurgence of an idea that failed to take hold across almost one hundred years of secondary 
education philosophy and practice. At its inception at the turn of the century, the structure of 
publicly-funded vocational education was heatedly debated by unusual alliances of educators. 



federal and state governments, business leaders, economists, and social reformers (Kantor & 
Tyack, 1982). Two primary camps emerged. On one side were those who favored an efficient 
mechanism of separate schools that would train students for their probable futures as entry-level 
workers in specific occupations within industry, commercial enterprises, agriculture, or the 
home (Grubb, Davis, Lum, Plihal, & Morgaine, 1991; Kantor & Tyack, 1982). This style of 
vocational education was seen as a positive solution to the steadily growing population of 
immigrants and the poor, who were seeking a public education, but were deemed not likely to 
pursue vocations requiring college preparation. On the other side of the debate were those who 
believed that a broad-based, applied academic curriculum would best prepare all students as 
citizens in a democracy (Boesel, Rahn, & Deich, 1994; Grubb et al., 1991; Kantor & Tyack, 
1982). 

The Smith-Hughes Act of 1917 formally and firmly established a system of vocational 
education as envisioned by the proponents of occupationally-specific high school training for 
non-college bound students (Berkell & Gaylord-Ross, 1989; Kantor & Tyack, 1982). However, 
as early as the 1930s, the structure of vocational education came under attack from social 
reformers and policy makers. The Russell Report of 1938 criticized the narrow focus of 
vocational education and urged adoption of a more flexible curriculum (Grubb et al., 1991; 
Kantor & Tyack, 1982). Federal initiatives in the 1960s and 1970s, including the 1963 
Vocational Education Act, also suggested that vocational education was overly narrow in scope 
and had become associated with limited educational opportunities and low-status occupations. 
Federal attempts to promote a broader approach to vocational training and to encourage the 
integration of vocational and academic curriculum failed to move the established system (Grubb 
et al., 1991; Kantor & Tyack, 1982). 




9 



Gray (1994) argued that despite the enduring nature of vocational education as an 
institution, there have been important shifts during its history. These shifts occurred as a result 
of gradual changes in parents' and students' beliefs about available and financially-rewarding 
occupations. At its inception and for several decades, parents and students supported the basic 
vocational programs in industry, agriculture, home economics, and commercial education 
because such training matched their perceptions of post-high school job opportunities. However, 
as parents and students saw these types of jobs become less available and desirable, more 
students began to sample a variety of courses that broadly fit into high school graduation 
requirements, but were not necessarily focused on a specific career. By the 1980s, a college- 
degree was perceived as the best way to have a better financial future, and more students took 
academic course work to gain the credits necessary for college entry. For many students, this 
often meant an incoherent series of general curriculum courses that failed to prepare them 
adequately for work or college (Barton, 1990; Gray, 1994; Kazis, 1993; Plihal, Johnson, Bentley, 
Morgaine, & Liang, 1992). Recently, programs that offer students both technical and academic 
skills again have been offered as a solution to this problem. 

Current Rationales for Integration of Academic and Occupational Curricula 

Integration of academic and occupational instruction has gained renewed attention 
because, as in previous years, it is a key component of federal school-to-work initiatives. 
Additionally, a variety of interest groups perceives that such curricular integration supports and 
is supported by other current education reform efforts, such as teacher professionalization, 
schools of choice, accountability, restructuring, and teaching reforms that emphasize challenging 
material, problem-solving, and cognitive skills (Andrew & Grubb, 1992; Boesel & McFarland, 
1994; Grubb et al., 1991; Hoachlander, 1994; Little, 1992; Stem, Finkelstein, Stone, Latting, & 
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Domsife, 1994). Thus, although integration is largely an effort to improve the quality of 
vocational education, it has become a centerpiece of many secondary education reforms. 

Several rationales for integrating academic and occupational curricula have been 
advanced. First, some critics of secondary high school curricula suggest that a general secondary 
education is not functional in design nor is it perceived as relevant by students (Grubb et al., 
1991). Proponents of integrated curricula argue that applied, competency-based courses that 
incorporate challenging academic material inherently would be more relevant and interesting to 
students. Because students would use real-world materials to solve problems in context, they 
would be more likely to understand the connections between abstract academic concepts and 
real-world work (Lankard, 1996). 

Second, there is concern that exposure to traditional vocational or general academic 
curricula alone does not sufficiently promote student academic skills and employment outcomes. 
Bodilly, Ramsey, Stasz, and Eden (1993) found in their case studies of schools that were early 
implementers of integrated curricula that many of these schools hoped to enhance students’ 
academic skills by infusing more academic learning into vocational courses. This belief that 
integrated curricula will promote student achievement has its basis in cognitive research that 
suggests applied instruction is a powerful tool to improve student attitude and engagement and 
to encourage independent problem-solving (Bodilly, et al., 1993; Grubb et al., 1991; Kerka, 

1986; Little, 1992). 

Third, it has been suggested that the workplace is shifting from an emphasis on mass- 
production work to "high performance" work, which means that workplaces and job duties will 
undergo frequent changes and lower level workers may have increased responsibilities (Bailey, 
1995). Although there is debate as to the rapidity and pervasiveness of this shift, proponents of 
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integrated curricula argue that both business and students benefit when students are taught to 
employ independent problem-solving, become familiar with workplace contexts, and are 
exposed to current technologies and business practices (Bailey, 1995; Lankard, 1996). 

Fourth, Bodilly et al. (1993) found that the desire to eliminate tracking was another 
motivating factor for schools that had adopted integrated curricula approaches. Critics of 
tracking and stratified curricula believe the split between vocational and academic programs 
contributes to unequal educational access and outcomes (Little, 1992; Oakes, Selvin, Karoly, & 
Guiton, 1992; Plihal et al., 1992). It is important to note that the predominant equity argument 
centers on eliminating the educational program divisions between college-bound and non- 
college-bound students. However, the equity rationale also encompasses assurances of access to 
programs by students with disabilities and other special populations. 

Students with Disabilities: Access to Integrated Curricula 

A policy overlap between the IDEA (1992), the Perkins Act (1990), and the School-to- 
Work Opportunities Act (1994) has emerged that emphasizes equity in secondary programs for 
students with disabilities (Moore & Waldman, 1994). The Perkins Act (1990) requires that 
students with disabilities and other special populations have access to vocational programs, and 
a central feature of these programs under the Perkins Act amendments is the integration of 
academic and occupational curricula. 

Prior to the passage of the Vocational Education Act of 1963, few vocational education 
programs for students with disabilities existed (Wagner, 1991). The 1963 Act and its 1968 
amendments that established funding specifically for special populations increased opportunities 
for students with disabilities to participate in secondary vocational education (Phelps & 
Wermuth, 1992; Wagner, 1991). Currently, the 1992 Carl D. Perkins Vocational Education Act 
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regulations require that regular vocational education programs support special education 
transition initiatives (National Transition Network, 1993). Even though funding set aside 
specifically for special populations has been replaced by targeted funding for localities with high 
concentrations of special needs groups, vocational education administrators continue to devote 
funds to promote involvement of special populations, including students with disabilities, in 
regular vocational education programs (Boesel & McFarland, 1994; Hudson, 1994). 

Like non-college bound regular education students, young adults leaving special 
education experience problematic outcomes, but to a greater degree than their peers without 
disabilities (Gaylord-Ross, Halloran, & Siegel, 1993; Hasazi, Johnson, Hasazi, Gordon, & Hull, 
1989; Johnson & Rusch, 1993; Wagner, Newman, D'Amico, & Jay, 1991). In addition to 
providing access to vocational education for students with disabilities, policy makers and 
practitioners concerned with the post-school outcomes of young adults with disabilities have 
implemented a variety of legislative initiatives and programs to facilitate adolescents' transition 
from special education programs into adult work settings (Halpem, 1994; Stodden & Leake, 
1994). The Individuals with Disabilities Education Act (IDEA) was amended in 1990 to include 
a mandate for individualized transition services for students with disabilities. This action built 
on earlier federal systems change efforts, and resulted in renewed efforts at the local and state 
levels to promote positive student outcomes (Gaylord-Ross et al., 1993; Wehman, 1992). 

Regular vocational education for students with disabilities has received additional support from 
least restrictive environment and transition provisions in the Individuals with Disabilities 
Education Act and from the Vocational Rehabilitation Act (Wagner, 1991). The work-related 
goals of regular vocational education are compatible with those articulated in the special 
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education literature and legislation (Cobb & Neubert, 1992), and vocational training is 
frequently cited as a "best practice" in special education (Kohler, 1993). 

Boesel, Hudson, Deich, & Masten (1994) noted that despite issues of limited access in 
some areas, special populations are over-represented in vocational programs. Although the 
number of students taking vocational education courses declined in the late 1980s (partially in 
response to increased state and district demands for more academic course-taking) there was no 
change in the vocational course-taking of students with disabilities. Students classified as special 
populations earned more vocational credits and tended to complete sequences of vocational 
courses. However, they took courses in the less technical areas of agriculture, occupational home 
economics, and trades. There were relatively few students from special populations in business, 
marketing, health, and technical education. This pattern was clear for students with disabilities. 

Although equal access by special populations to vocational programs has been a major 
outcome focus in the past, over-representation of students with disabilities in vocational 
education training that is less in demand in emerging labor markets suggests a different focus 
may be needed. The National Assessment of Vocational Education recommended that future 
reform efforts must improve the quality of occupational programs for special populations and 
provide assistance for the development and use of performance measurement systems that will 
adequately assess the impact of such programs (Boesel & McFarland, 1994; Hudson, 1994). 

It is at this juncture of program equity and program improvement, fueled by a variety of 
interests in regular and special education, that questions about the nature of integrated academic 
and occupational curricula for students with disabilities must be asked. Fundamentally, what is 
“integrated curricula” ? What benefits accrue to those who participate in integrated curricula? 



Does evidence exist to suggest that exposure to integrated curricula is equally beneficial for 

students with and without disabilities? 

~ Models and Components of Integrated Curricula 

Much has been written about integrating academic and occupational curricula within the 
last several years, primarily from a conceptual or best practices point of view. Plihal et al. 
(1992) found more than 90 descriptions of integrated academic/occupational curricula in 
elementary, secondary, and postsecondary education published before 1990; most within the 
preceding five years. Stasz, Kaganoff, and Eden (1994), in their search for empirical studies on 
integrated curricula, found slightly more than 100 publications - few of which were empirical - 
for the period between 1987 to mid-1992. In general, these reviews found substantial amounts 
of activity and interest, but not much clarity about purposes, practices, or outcomes for 
integration (Stasz et al., 1994). 

Organizational Contexts for Integration 

This diffuse effort may be a result of the fact that integration has been advocated and 
adopted by several different education reform movements (Andrew & Grubb, 1992; Lankard, 
1996; Little, 1992) and involves a variety of changes at the organization level as well as changes 
in teaching and learning practices. It is not a single, well-defined intervention, but instead 
constitutes a package of changes in a school's social context and pedagogical practices that may 
be modified to fit the needs and interests of the implementers. At the organizational level, 
integration implies changes and enhancements to schools, including: more local control, new 
resources and departmental changes to support collegial work, teacher training, easy access to 
knowledge resources, reconsideration of scheduling and tracking practices, a press for 
achievement, and sustained implementation efforts by key, committed staff (Andrew & Grubb; 
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1992; Little, 1992; Staszetal., 1992). Most current school reforms call for similar organizational 
changes, and in this regard, integration of curricula is comparable to other secondary education 
reforms. To further understand how integrated curricula is similar to, yet distinct from other 
reforms requires examination of proposals for (a) the organization and content of subject matter, 
and (b) classroom teaching and learning activities. 

Content blend: What is taught? 

Because of the entrenched content area focus of most high schools, subject matter is the 
primary organizing principle of current attempts to integrate curricula (Little, 1992). Typically, 
integration of academic and occupational curricula focuses on blending together traditional 
disciplinary subject matter. The “academics” most often refers to the college preparatory 
curriculum, especially math, science, English, and history (Plihal et al. 1992). “Vocational” 
means those courses that schools offer to prepare students for entry into specific jobs or to 
develop workplace attitudes (Gray, 1994; Plihal et al., 1992). 

Curricular integration is often described as being “horizontal” or “vertical” (Grubb et al., 
1991; Plihal et al., 1992). Horizontal alignment refers to integration across one level of the 
curriculum. For example, an academic and a vocational course may be linked together by 
overlapping or reinforcing material presented in each, using a common theme to guide content 
selection, or developing a new “fused” course that merges subject matter of the original courses 
(Andrew & Grubb, 1995; Plihal et al., 1992). Vertical alignment refers to course sequencing, 
usually across several disciplines or within a newly defined “broad fields” area (Grubb et al., 
1991; Plihal et al., 1992). Both types of curriculum integration have been observed in recently 
implemented models. Based on field studies of emerging programs, Grubb et al. ( 1991) 
identified eight implemented models of integrated curricula. Reid and Tsuzuki (1994), in their 



national review of local practice, provided examples of school changes and classroom activities 
for each of Grubb et al.’s eight implemented models. These models range from teacher 
adaptations of one course’s content to complex whole-school organizational reforms such as 
occupational magnets, in which all students engage in a sequence of courses within one or more 
occupational areas. 

Although comprehensive models such as career academies and occupational magnets 
exist, typical integration efforts have been on a smaller scale (Boesel, Hudson et al., 1994). The 
most prevalent types of integration to date are those that (a) incorporate basic academic skills 
into industrial-trade vocational classes or (b) replace general English, science and math courses 
with locally-adapted versions of curricula packages that infuse vocational material into 
academic content, such as Applied Communication, Principles of Technology, and Math for 
Technology (Boesel, Rahn et al., 1994; Lankard 1996; Plihal et al., 1992). Rarely in practice 
does integration also include issues of family or personal development, although this has been 
recommended (Copa, 1994). Most of these initial integration efforts are limited in two ways. 
Ideally, subject matter of integrated curricula encompasses (a) vocational content that is less job- 
specific, and (b) challenging academic content acquired through application. 

Expanding the vocational content domain. The traditional job-specific conception of 
vocational education is changing to the broader view of educating students through an “industry- 
related” or “occupational clusters” approach (Hoachlander, 1994). In this view, high school is 
considered an occupation-related endeavor for all students. The function of the high school 
curriculum is to prepare students for entry into their chosen vocation, whether that means 
graduating directly to the job market or continuing their education and training at the 
postsecondary level before entering into a profession. Further, most vocational students do not 



obtain a job in the field for which they trained (Boesel, Rahn et al., 1994). Therefore, proponents 
of integrated curricula suggest that in addition to, and typically before, learning job-specific 
skills students should develop “generic” skills and broad “industry-relevant” knowledge and 
skills; a process that may begin prior to high school (Bailey, 1995; Berryman, 1995; Boesel, 
Rahn et al., 1994; Copa, 1994; Grubb et al., 1991; Hoachlander, 1995; Plihal et al., 1992; 
STWOA, 1994). 

Generic skills include (a) basic skills such as reading, simple math, life skills, and 
prosocial behavior, (b) formal and informal complex reasoning skills, and c) work-related skills 
and attitudes such as self-management, cooperative skills, and “habits of thought” (Stasz et al., 
1992; Stasz, Ramsey, & Eden, 1995). These generic skills are similar to the “foundation skills” - 
basic skills, thinking skills, and personal qualities - proposed in the U.S. Department of Labor 
SCANS report (1991) that has been widely supported by industry and many vocational educators 
(Lankard, 1995). 

Industry-relevant knowledge and skills address a wide-range of content using a particular 
industry (e.g., transportation) as a specific context. Students acquiring industry-relevant 
knowledge might cover: structure and organization, history, technology, economics, human 
resources, public policy, health and safety, and environmental impact (Hoachlander, 1995). 
Typically, an emphasis on job-specific competencies would occur following exposure to generic 
and industry-relevant skills in both the classroom and community. 

Acquiring challenging content through application. Boesel, Rahn et al., (1994) suggest 
that redefining and blending content alone is not sufficient; good student outcomes are 
dependent upon developing general cognitive ability as well as acquiring relevant basic skills 
and occupational knowledge. In studies of academic course-taking, cognitive skill and post- 



school outcomes, cognitive skill appears to be related to positive post-school outcomes for 
students who do not proceed to college (Boesel, Rahn et al., 1994; Gamoran, 1995). Integrating 
academic and occupational curricula is seen as a way to emphasize the importance of cognitive 
skill development and to expose more students to challenging academic content (Boesel, Rahn et 
al., 1994; Plihaletal., 1992). 

Proponents suggest that embedding abstract academic concepts into real-world 
applications would enable students to encounter and utilize information that was not previously 
available to them in the traditional hierarchical arrangement of decontextualized academic 
courses (Andrew & Grubb, 1992; Bailey, 1995; Bodilly et al., 1993; Bottoms, Presson, & 
Johnson, 1992; Stem et al., 1994). This concept of “applied learning” implies (a) connections 
between “school” work and “real” work are made explicit causing school learning to become 
more meaningful to more students, (b) problem-solving skills are regularly put into practice, and 
(c) symbolic or abstract theory is made accessible through “hands-on” projects that use real 
world materials. In a fully integrated curriculum, students would be required to use both 
principles and practices, rather than relying on the rote learning that is typically required in high 
school courses (Bailey, 1995; Berryman, 1995; Boesel, Hudson et al., 1994; Bottoms et al., 

1992; Karweit, 1994; Plihal et al., 1992; Stem et al., 1994). 

Teaching and Learning; How is it Taught? 

These changes to the subject matter of high school curricula suggest changes to teaching 
and learning activities within classrooms. Typically, efforts to promote the integration of 
academic and occupational curricula are tightly coupled with calls for pedagogical reform. 
Generally, teaching and learning are expected to be more activity-based and student-centered 
(Andrew & Grubb, 1992; Bailey, 1995; Bodilly et al., 1993; Plihal et al., 1992; Stasz et al, 1992; 



1995). To support this approach, the literature on integrated curricula often references cognitive 
science principles of ideal learning environments and the idea of “cognitive apprenticeships” 
(Berryman, 1995; Grubb, 1995; Karweit, 1994; Staszetal., 1992; 1995). 

These principles were summarized by Collins, Brown, and Newman (1989) in their 
examination of three models of cognitive apprenticeship that promoted students’ academic 
achievement. In these models, skills and knowledge were taught in the context of meaningful 
tasks. Teachers used modeling, coaching, and fading as instructional strategies. Thinking and 
problem-solving skills were externalized by teachers to permit student observation of experts 
and other learners and to facilitate students' self-correction. Students learned global skills before 
specific skills. The learning environments featured elements of cooperation and competition. 

Similar principles were identified also by Brophy & Alleman (1991) in a literature 
review supporting their conceptual analysis of learning activities. They suggested that ideal 
learning activities have goal relevance, appropriate level of difficulty and high motivational 
value, incorporate multiple goals, address higher-order thinking and life applications, and 
require students to connect declarative knowledge with procedural knowledge. Teachers 
implemented these activities by introducing and scaffolding content, supporting independent 
student work, and providing opportunities for debriefing, reflection, assessment, and feedback. 

Stasz et al. (1992, 1995), based on their qualitative studies of classrooms, believed that 
classes resembling this type of “ideal” learning environment would support the acquisition of 
generic and integrated academic/occupational knowledge and skills. They described successful 
classrooms as incorporating the following elements: (a) "situated learning," learning in context 
using real world applications that require both subject-specific and generic skills; (b) a "culture 
of practice," in which students leam how experts solve problems; (c) intrinsic and extrinsic 



“motivation’' as reported by students and teachers, and (d) "cooperation,” observable as self- 
managing groups, students acting as consultants to other students, and frequent role changes 
between assistant and learner. 

Stasz et al. (1992) noted that in these classrooms teachers became more like coaches, 
guides, or masters working with apprentices, and made limited use of lecturing or teacher- 
directed discussions. The teachers often engaged in teaching that made use of unplanned, 
naturally occurring events and provided one-to-one tutoring by circulating and visiting groups, 
focusing on students who needed assistance. Parallels between these elements of "integrated 
pedagogy" and the workplace have been suggested. For instance, successful workers often learn 
in context, use problem-solving skills, engage in independent, active learning across fields, and 
must function cooperatively with others or as members of teams (Bailey, 1995; Brown, Collins, 
& Duguid, 1989). 

Central Features of Integrated Curricula 

As envisioned by practitioners and proponents, integrated curricula puts into place a high 
school experience that may be described as functional, challenging, and applied (Lankard, 

1996). Although, as Grubb (1995) acknowledged, each implementer must answer difficult 
questions regarding the degree to which different aspects of the contextual, curricular and 
pedagogical practices will be emphasized, certain central features are evident. First, in an 
integrated curriculum a student will experience vocational content that emphasizes generic and 
general work-related skills and knowledge; vocational experiences are neither omitted nor only 
job-specific. Second, academic and occupational content are blended in a way that draws 
attention to school and work connections; students work with knowledge of both academic 
principles and real world practices. Third, students are more often engaged in applied activities 
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that require them to be problem-solvers instead of passive receivers of knowledge. Fourth, 
teachers spend less time as dispensers of knowledge and more time guiding and supporting 
student work. Finally, teachers and students are engaged in cooperative activities. 

Features of Integrated Curricula and Special Education 

Examination and discussion of the constructs comprising integrated 
academic/occupational curricula have occurred primarily in the area of regular education (e.g., 
Plihal et al., 1992). However, indirect support for some of these features exists in the special 
education literature. Wagner (1991) reported that participation in vocational education is related 
to positive employment outcomes for students with disabilities, especially those with mild 
disabilities. Students with disabilities who took vocational courses in high school were more 
likely to get employment, and if they took four or more vocational credits, they also improved 
their earnings. Other studies have substantiated that students with disabilities who participated in 
vocational education during high school had more positive postschool outcomes when compared 
to students with disabilities who had not taken vocational courses (Kohler, 1993). However, the 
goals, curriculum content, and instructional methods of the vocational courses that related to 
positive effects have not been systematically assessed. It is not clear whether a broad 
occupational, skills cluster approach has a greater impact than a narrower job-specific skills 
training approach. Support for both approaches exists (Kohler, 1994; Wagner, 1991). 

Generally, the special education literature suggests that secondary students with 
disabilities should have access to “functional”, individualized curricula that address academic, 
vocational, social, and independent living skills (Cipani & Spooner, 1994; Clark, Field, Patton, 
Brolin, & Sitlington, 1994; Edgar & Polloway, 1994; Falvey, 1989; Halpem, 1994; Kohler, 
DeStefano, Wermuth, Grayson, & McGinty, 1994; Smith & Puccini, 1995). Functional curricula 



are "outcome-oriented”; addressing skills a student needs in his or her current, next, and future 
environments (Fredericks & Brodsky, 1994). A key issue is that individualization of the 
curriculum is a function of both the student's disability and personal goals. Students with 
disabilities are an extremely varied group. Disabilities may be mild or severe; cognitive or 
physical; require modest accommodation or intensive supports. Students' postsecondary goals 
may address college education, employment, residential options, and community participation. 
Depending upon these student characteristics and goals, a “functional” curricula may contain 
more or less academic content delivered in a variety of ways. 

For some students, the academic content of the general curriculum is not functional, and 
teaching “basic” academic skills in the context of critical independent living and specific 
employment skills may be reasonable and relevant. A content blend approach to high school 
curricula makes sense for students with disabilities if that means providing a range of academic, 
vocational, and other life skills content within an outcome-oriented framework that recognizes 
both postsecondary academics and employment as meaningful goals for many students with 
disabilities. Also, blending academic and occupational courses and programs may facilitate 
meaningful participation and physical placement of students with disabilities in regular 
education classrooms by making academics more available and applied (Phelps, 1992). 

A strength of the special education field is its research-based development and 
implementation of instructional modifications to meet the needs of diverse learners (Cipani & 
Spooner, 1994; Edgar & Polloway, 1994; Pugach & Warger, 1993). Instructional support 
models, such as teaching learning strategies, tutoring, cooperative learning, and collaborative 
teaching, are typical instructional strategies for students with mild disabilities in secondary 
education (Edgar & Polloway, 1994). For students with severe disabilities, systematic data-based 



instruction characterized by frequent opportunities to learn, fading of behavioral interventions, 
use of natural cues, and training for generalization is considered best practice (Falvey, 1989). 
Within special education practice, a variety of instructional strategies must be employed to meet 
the wide-ranging needs of students with disabilities. There is a need to explore the different 
types of instructional strategies and supports that promote productive engagement of students 
with different disabilities in applied academic courses. 

In general, special education professionals have taken the position that special education 
must be part of secondary education restructuring and can best do so by advocating for 
improvements to and expansion of curriculum and transition services on behalf of all students 
(Benz & Kochhar, 1996; Halpem, 1994; Polloway, Patton, Epstein, & Smith, 1989; Patton et al., 
1996). Patton et al. (1996) suggested that expansion of curricular options in regular education, 
especially in the areas of vocational and life skills, is necessary. Otherwise, efforts to promote 
inclusion will result in placements in regular education classes that are nonfunctional for many 
students with disabilities. Additionally, Bodilly et al. (1993) suggested that integrated curricula 
may improve school transition for regular education students because successful 
academic/occupational integration requires using planning partners, developing a transition- 
specific curricula, examination of transition services, and enhancing teaching credentials and 
certification. This suggests that an ancillary system-level outcome of integrating curricula in 
regular education could be the enhancement of existing transition services for students with 
disabilities that have become a mandate for practice through the IDEA. 

Previous Research on Effects of Integrated Curricula 

At this time, the evidence that integrated curricula may benefit students without 
disabilities is indirect; the greatest support comes from evaluations of comprehensive programs. 



such as career academies and magnets. These programs seem to promote student achievement 
and persistence. Empirical evidence of effects for students wilh disabilities is almost 
nonexistent. Although model integrated programs for students with disabilities exist, limited 
generalizable information on outcomes of these programs is available. 

Three recent reviews of the literature on integrated academic/occupational curricula have 
examined practices or related outcomes (Plihal et al., 1992; Stasz et al., 1994; Stem et al., 1994). 
Additionally, one reviewer examined studies of contextual learning (Karweit, 1994). Two of 
these reviews were conducted on behalf of the most recent National Assessment of Vocational 
Education (Karweit, 1994; Stasz et al., 1994). 

Previous reviewers of the literature on integrated academic/occupational curricula have pointed 
out that most studies inadequately defined interventions, and relatively few provided any 
empirical evidence of the effects of integrated curricula. Common methodological problems of 
empirical studies on integration included failure to adequately define program components and 
dependent variables, limited or no information regarding degree or fidelity of implementation, 
infrequent use of comparison groups or statistical controls, and limited reporting of basic 
statistical results. There is some evidence that comprehensive programs such as magnets and 
academies that have an integrated academic/occupational focus may provide academic benefits 
to some average and low-achieving students considered to be at risk for dropping out (e g., 

Crain, Heebner, & Si, 1992). However, results are inconsistent across program sites and student 
groups. Studies have not examined postschool outcomes or vocational competencies. To date, 
studies of contextual learning offer only limited and indirect evidence of benefits of such 
teaching practices for secondary students. 
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Recent descriptive studies of schools, districts and states have provided additional 
information regarding important features of policy environment, school context, and classroom 
pedagogy that support integration of academic and occupational curricula (Bodilly et al., 1993; 
Stasz, et al., 1992). Researchers have identified “exemplary” programs for students with 
disabilities that incorporate integrated curricula (Matias, Maddy-Bemstein, & Kantenberger, 
1995). Additionally, researchers have begun to identify teacher and administrator views 
regarding potential outcomes of integrated curricula (Finch, Schmidt, & Faulkner, 1994) or have 
documented program-reports of general outcomes (Matias et al., 1995). Two of the studies 
(Matias et al., 1995; Stasz et al., 1992) reported on the participation of students with disabilities 
in programs or classes that featured content or instructional elements of integrated curricula. 
Although each of these studies implied that integrated curricula may promote positive student 
outcomes, none specifically attempted to demonstrate effects. It seems clear that engagement of 
students with disabilities in classes or programs that provide integrated academic/occupational 
curricula is possible, but whether integrated curricula may benefit students with disabilities 
cannot be concluded from these studies. 

Recent evaluations of programs that include an integrated curricula component have 
ranged from studies of large scale model program replications to evaluations conducted by a 
teacher-researcher in a single school. These evaluations have provided only limited evidence to 
support the contention that engagement in integrated curricula leads to benefits for students. The 
principle reason for this situation is that integrated curricula is typically one component among 
many in a complex set of interventions within programs, and these programs may be one of 
many being promoted by reformers. Isolating the effects of integrated curricula has been 
difficult also because only a few studies have examined the nature or level of implementation 



(e.g.„ Stem, Raby, & Dayton, 1992) or have attempted to systematically describe elements of 
implementation (e.g., Penn, 1992; Raber & Merchlinsky, 1995). Further, as found by previous 
reviewers, these evaluation reports -- with the exception of the career academy studies (Stem et 
al., 1992) - generally failed to provide information regarding the types of data analyses used and 
basic statistical results needed to evaluate program effects. Information about the effects for 
students with disabilities is similarly limited. Although at least one (Penn, 1992) included 
students with disabilities and two other programs (Project Coffee, as reported by Leutheuser, 
1994, and the career academies) may have, none of the studies provided sufficient detail to 
determine the types of disability groups that may have been represented. The Southern Regional 
Education Board studies (Bottoms, 1995; Bottoms & Presson, 1995) provided interesting 
baseline data on a large group of students using a national comparison group. Unfortunately, 
without additional detail regarding methodological aspects of the studies, interpretation of future 
evaluation results will be difficult. In general, recent program evaluations have suggested that 
programs with an integrated curricula component may contribute to student persistence, 
academic achievement, and postsecondary engagement. The benefits of integrated curricula 
itself are not clear from these studies. 

The Problem of Assessing Implementation 
Determining effects of integration efforts has not been straightforward for several 
reasons. Integrating academic and occupational curricula involves changes in both content and 
pedagogy; multiple classroom features must be defined and measured. Integration may occur 
within or across courses; units of analysis may be students, classes, groups of classes or schools. 
Integration may be coupled with other school organizational changes; effects may be 
confounded by multiple component interventions. Programmatic changes may take several years 



to fully implement and some student outcomes of interest occur post-graduation; studying 
effects may require a longitudinal approach. These issues pose methodological challenges for 
future research on integrated curricula. 

Future research on integration of academic and occupational curricula for students with 
disabilities can strengthen understanding of the processes and effects of integrated curricula by 
attending to and accounting for the following: (a) participation of special populations, including 
those with disabilities, (b) clearly defined multiple measures or indicators of program and 
curriculum components, degree of implementation, and student and organizational outcomes, (c) 
consideration of measures that reflect the underlying values and causal-models of local 
implementers and participants, (d) developmental or longitudinal approach to accommodate or 
control for levels of implementation, and (e) complete reporting of quantitative and qualitative 
results in multiple formats accessible to practitioners, policy makers and researchers. 

Initially, a greater understanding of what happens in classes and schools that integrate 
academic and occupational curricula is needed. Few of the existing studies on outcomes of 
integrated curricula also provided information about specific features of implementation. What 
type of content is actually delivered in integrated curricula classes and how? To what extent are 
elements deemed critical for many special education students (e.g., life skills) provided in 
integrated classes? To what extent are features of integrated curricula already present in the 
variety of classroom environments experienced by special education students? Are instructional 
strategies typically used in special education similar to those used in integrated classes? 

Porter (1991) and Kohler (1993) have suggested that indicators of actual classroom 
practice are needed to overcome this weak knowledge base related to effects of school programs. 
In the past, outcome assessment at the program or systems level has relied on broader measures 
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of student coursework such as student enrollment and number of credits in particular subjects 
(McDonnell et al., 1990) or narrower measures related to specific interventions (e.g., Hughes et 
al., in press). How school services impact student outcomes cannot be assessed adequately until 
there are sound indicators about the curriculum and instruction that schools actually deliver in 
the classroom. At this time, the development of the needed methodology has received little 
attention (Porter, Kirst, Osthoff, Smithson, & Schneider, 1993) and use of more sophisticated 
indicators is limited to national studies (McDonnell et al., 1990). 

Research Objectives 

Valid indicators of classroom implementation are needed for (a) future research on 
secondary-education reforms, (b) investigation of classroom factors associated with positive 
student outcomes, and (c) exploration of the validity of current conceptual models of curriculum 
and instruction for secondary students with and without disabilities (Kohler, 1993; McDonnell et 
al., 1990; Porter, 1993). 

The primary objective of this study was to validate measures of classroom practice that 
would be sensitive to the level of integration of academic and vocational curricula and that 
might be applied to a variety of secondary courses including those experienced by students with 
disabilities. To further examine validity, secondary objectives of the study were to assess (a) 
reliability of the measures and instruments and (b) relevance of resulting information. If 
validated, the measures and instruments would be useful in future evaluations regarding 
implementation of integrated curricula and research on associated outcomes at the student, 
classroom, school, or district level. 
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CHAPTER II 



METHODS 



Research Design 



This study used systematic observation methods combined with brief teacher and 
administrative surveys to examine the construct validity of classroom practice measures related 
to the integration of academic and occupational curricula. The measures of interest were 
sampled in four different types of classes using multiple instruments. Analyses focused on 
discrimination and convergence of evidence (Campbell & Fiske, 1959; Morris, Fitz-Gibbon, & 
Lindheim, 1987). 



Two schools were selected that participate in the Southern Regional Education Board's 
(SREB) High Schools that Work initiative. A central feature of High Schools that Work is an 
emphasis on integration of academic and occupational curricula (Bottoms, Presson, & Johnson, 
1992). According to the Southern Regional Education Board and school administrators, the 
selected high schools had implemented integrated curricula in several courses for three to four 
years and also had traditional high school courses. Thus, a purposive sample of classes from 
each school could be created to reflect groups representing traditional academic, traditional 
vocational, integrated curricula, and special education academic classes. These classes 
encompassed a variety of subjects (e.g., mathematics, cosmetology, applied communications), 
grades (i.e., 9th through 12th), and academic levels (e.g., remedial, honors). 

Initially, the sample included 38 classes conducted by 31 teachers. Five teachers taught 
both an integrated curricula class and a traditional academic class. In order to maintain distinct 
groups, only one course's data were used for each teacher. Of the five classes eliminated in this 



Sample 





30 



55 



way, three were academic classes and two were integrated curricula classes. The choice of which 
classes to maintain in the sample was made based on either (a) the teachers' reputations with 
school administrators as innovators with integrated curricula, in which case the integrated 
curricula class was selected, or (b) because the teacher reported that the materials used and 
activities conducted during the study period were not representative for his or her typical 

t 

integrated curricula class, in which case the academic class was selected. Two special education 
resource teachers originally participated for two class periods each to ensure a variety of special 
education academic subjects in the sample, but one of each teachers' classes was dropped from 
the final sample to maintain similar group sizes. Two teachers, one vocational and one special 
education, dropped out early in the study because they could not regularly complete the required 
Daily Logs. Two teachers, one academic and one integrated curricula, were eliminated from the 
final sample because they questioned the value of their data. The integrated curricula teacher 
reported that he believed his class was actually academic despite its designation as integrated by 
the administration. The academic teacher reported that she did not believe her logs were valid 
because she was “never sure how to fill out the [daily logs]”. The final sample consisted of 27 
classes: six regular education academic, seven regular education vocational, seven regular 
education integrated, and seven special education academic. Figure 1 displays the course titles. 
Classroom Demographics 

The following classroom demographic variables were explored: class size, percent 
female, student classifications (i.e., special populations: low income, limited English 
proficiency, academically talented, special education), academic and technical rigor, and student 
graduation path. These variables are presented primarily for descriptive purposes, because they 
are frequently mentioned in the integrated curricula literature and also in related legislation. 
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“FAMS = Ford Academy of Manufacturing Sciences 

b Lifeskills = self-contained class for students with mental retardation; observed during academic 
periods 

Figure 1. Course Titles 



Both the literature and law (e.g., STWOA) emphasize equal access to school-to-work activities 
and high academic expectations for special populations. Student classifications were determined 
by the schools. 

Two factor analysis of variance (ANOVAs) tests for school (School 1, School 2) and 
group (academic, vocational, integrated curricula, special education) were conducted to 

32 



0 




57 



determine if significant interactions occurred that might reflect unintended differences among 
classes in the sample. If significant interactions existed, i tests were used to identify the 
interactions. If no significant interactions were evident, main effects for school or group were 
explored. If significant differences existed, Bonferroni post hoc tests for multiple comparisons 
were conducted and means examined to identify differences. The means and standard deviations 
for classroom demographics by group are displayed in Table 1 and reviewed individually below. 

Class Size. The average class size was 17. 15 (sd = 7.75, range 6 to 32). There was 
significant interaction for class size, F (3, 19) = 7.81, p = .001. Follow-up f tests demonstrated 
that vocational classes at School 1 (M = 12.67, sd = 5.51) were significantly smaller than those 
at School 2 (M_= 25.75, sd = 4. 19), t (5) = -3.60, p = 0.02, and integrated classes also were 
significantly smaller at School 1 (M = 1 4.00, sd = 3. 1 6) than School 2 (M = 24.33, sd = 5. 1 3), t 
(5) = -3.33, p = 0.02. There were no significant differences in class sizes in academic classes, f 
(4) = 1.75, ns (School 1, M = 25.33, sd = 5.03; School 2,M= 17.67, sd = 5.69), or special 
education classes, 1 (5) = 0. 14, p = ps (School 1 , M = 9.33, sd = 3.5 1 ; School 2, M= 9.00, sd = 
2.94). 

Percent females. There were no significant interactions for percent females in classes (M 
= 48.18, sd = 27.22), E (3, 19) = 0.07, ns- There were no main effects for percent females in 
classes at School 1 (M = 54.02, sd = 28. 18) and School 2 (M = 42.75, sd = 26. 14), E (1, 19) = 
1.02, ns. Neither were there significant differences between class types, E (3, 19) = 0.71, ps. 

Student classification. There were no significant interactions for percent students 
classified as low income (M = 40.57, sd = 19.66), E (3, 19) = 1.24, ps. Also, there were no 
significant differences in percent students classified as low income for School 1 (M = 

38.68, sd = 24.98) and School 2 (M = 42.33, sd = 13.80), E(l> 19) = 0.06, ps, or by group. 



Table 1. Classroom Demographics 



Class Types 





Academic 
(n = 6) 


Vocational 
(n = 7) 


Integrated 

(n=7) 


Special 

Education 

(n = 7) 


Variable 


M (SO) 


M (SD) 


M (SD) 


M (SD) 


Enrollment 


21.50 (6.38) 


20.14 (8.24) 


18.43 (6.66) 


9.14 (2.91)*** 


Males (%) 


37.88 (10.77) 


49.50 (40.64) 


58.29 (26.37) 


59.63 (21.13) 


Females (%) 


62.12 (10.77) 


50.50 (40.64) 


41.71 (26.37) 


40.37 (21.13) 


Classification (%) 
Low Income 


32.81 (13.52) 


48.74 (14.90) 


29.88 (21.02) 


49.76 (22.12) 


Limited English 


7.78 (19.05) 


1.37 (2.36) 


2.38 (6.30) 


0.00 (0.00) 


Honors 


34.97 (42.40) 


2.93 (5.01) 


40.32 (43.61) 


0.00 (0.00) * 

** 


Special 

Education 


2.95 (4.63) 


16.04 (17.13) 


7.34 (8.12) 


97.62 (6.30) * 


Rigor 

Academic 


3.00 (1.67) 


3.14 (0.90) 


3.57 (0.54) 


1.71 (1.25) * 


Technical 


2.00 (0.89) 


4.43 (0.79) 


4.29 (0.76) 


2.14 (1.46) * 


Academic Path (%) 


81.67 (26.58) 


4.5 (5.61) 


52.86 (40.71) 


6.14 (12.42) * 



"""♦Significant interaction for school and group, p < 0.05 
"■♦Significant difference between schools, p < 0.05 
♦Significant difference between groups, p < 0.05 



F (3, 19) = 2.08, ns. Also, there were no significant interactions for percent students classified 
as limited English proficient (M = 2.70, sd = 9.42), F (3, 19) = 0.91, ns. Neither were there 
differences between School 1 (M = 5.28,sd= 13.30)andSchool2(M = 0.31,sd = 1.16), E (1, 
19) = 1.93, ns, or between groups, E (3, 19) = 0.82, nS- 
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There were no significant interactions for school and group related to the percent of 
students classified as “ honors ” or academically talented (M = 18.98, = 33.74), F (3, 19) = 

0. 1 1, ns- However, significant differences were apparent between School 1 (M = 35.00, sd = 
42.42) and School 2 (M = 4. 1 1, §d= 10.99), F (1, 19) = 8.37, p = 0.009, and between groups, F 
(3, 19) = 4.22, p = 0.02. School 1 classified more students as honors than School 2. Also, classes 
designated as academic or integrated curricula enrolled more students classified as honors than 
vocational or special education classes. 

There were no significant interactions for school and group related to percent students 
classified as special education (M = 32.02. sd = 41.02), E (3, 19) = 0.35, n$. Likewise, there were 
no differences between percent special education students in School 1 classes (M = 27.41, sd= 
40.27) and School 2 classes (M = 36.31, sd = 42.75), E(l, 19) = 0.60, ns- However, there were 
significant differences by group, E (3, 19) = 1 10.05, p < .001. As might be expected, special 
education classes had higher percentages of special education students than academic, 
vocational, or integrated curricula classes. Students in the special education classes included 39 
with mental retardation (all but 3 in self-contained “Lifeskills” classes), 20 with learning 
disabilities (all in the resource classes), 2 with behavior disorders, and 2 with health or sensory 
impairments. Students classified as special education were included also in several of the regular 
education classes. Two of six academic, six of seven vocational, and four of seven integrated 
curricula classes enrolled students with disabilities. Of the 19 special education students in 
regular education classes, 9 had learning disabilities, 4 had mental retardation, and 2 each had 
language impairments, behavior disorders, or health or sensory impairments. The maximum 
number of students with disabilities in any single regular education class was three in academic, 
seven in vocational, and six students in integrated curricula classes. 



Rigor. Teachers rated the academic and technical (vocational) rigor of their classes 
compared to all other classes in the school using a scale of 1 (least rigorous) to 5 (most 
rigorous). There were no significant interactions for academic rigor (M = 2.85, sd = 1.29), F (3, 
19) = 1.32, ns, or technical rigor (M =3.26, sd = 1.51), F (3, 19) = 2.92, ns- There were no 
significant differences between schools on classes' academic rigor, F (1, 19) = 0.01, ns (School 
1, M = 2.92, sd = 1.44; School 2, M = 2.79, sd = 1.19) or technical rigor, F (1, 19) = 0.003, ns 
(School 1, M = 2.79, sd = 1 .84; School 2, M = 3.21,sd = 1.19). However, class types differed 
significantly on academic rigor, F (3, 19) = 3.52, p = 0.04, and technical rigor, E (3, 19) = 13.78, 
E < 0.001. Integrated curricula classes were rated as significantly more academically rigorous 
than special education classes. Vocational classes and integrated curricula classes were rated as 
having significantly more rigorous technical content than either academic or special education 
classes. 

High school path. Teachers were asked to estimate the percentage of students in their 
class who belonged to particular curriculum or graduation paths. (The state recently began to 
require students to select an academic or vocational/technical path, but only 9th grade students 
would have selected paths at the time of the study.) There were no significant interactions 
related to teachers' estimates of the percent students in the academic path (M = 35.77, sd = 
40.62) E (3, 19) = 2.70, ng. There was no significant difference between teachers' estimates in 
School 1 (M = 44.62, sd = 46.70) and School 2 (M = 26.92, sd = 32.97), E(l, 19) = 2.41, ns. 
Teachers' estimates did differ significantly by group, E (3, 19) = 17. 10, p < 0.001. Academic and 
integrated teachers were more likely to estimate that a higher percentage of their students were 
in academic paths than either vocational or special education teachers. 
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Teacher Experience 



Teachers' years of experience were analyzed in the same manner as the classroom 
demographics. Variables examined included: total years teaching experience, years experience 
teaching the course involved in the study, other professional experience outside the education 
field, and educational attainment. Teachers' experience by group is displayed in Table 2. 

Total years teaching. There were no significant interactions related to teachers' total 
years teaching experience, F (3, 19) = 1 .87, ns. Teachers at the two schools did not differ 
significantly on teaching experience (School 1, M = 12.69, sd = 8.08; School 2, M = 16. 18, sd = 
10.26),F(1, 19)= 1.67, ns. Teachers' experience did vary significantly by group, F (3, 19) = 
3.90, p = 0.03. Vocational teachers had significantly more years of teaching experience than the 
special education teachers. 



Table 2. Teacher Experience 



Class Types 





Academic 

(n=6) 


Vocational 

(n=7) 


Integrated 
(11=7) 


Special 

Education 

(n = 7) 


Variable 


m sd 


M 3D 


M 


SD 


M 3D 


Total Years 
Teaching 


16.00 (12.05) 


21.43 (5.03) 


13.86 


(6.47) 


6.93 (7.64) 


Years Teaching 
Course 


6.67 (6.86) 


17.57 (3.95) 


3.43 


(1.99) 


5.71 (8.02) 


Total Years Other 
Profession 


2.17 (3.92) 


20.29 (18.55) 


2.00 


(2.76) 


4.00 (5.42) 



*Significant difference between groups, p < 0.05 



Course Experience. There were no significant interactions for the number of years 
experience that teachers had with the specific course included in the study, M = 8.41, sd = 7.77, 
F (3, 19) = 3.07, ns. There was no difference noted by school, F (1, 19) = 0.002, ns. Teachers did 
have significantly different years experience with specific courses by group, F (3, 19) = 10. 13, p 
< 0.001. Vocational teachers had more experience teaching their course than academic, 
integrated curricula, or special education teachers had with teaching theirs. 

Other professional experience. A similar pattern of differences was noted for teachers' 
other professional (not teaching) experience (M = 7.50, sd = 12.55). There were no significant 
interactions for school and group, F (3, 19) = 0. 18, ns. Teachers at different schools did not have 
significantly different years of other professional experience (School 1, M= 7.85, sd = 12.77; 
School 2, M = 7. 1 5, sd = 12.83), F(l, 19) = 0.28,nS- However, vocational teachers had 
significantly more years other professional experience than all other types of teachers, F (3, 19) 

= 4.42, p = 0.02. 

Educational Attainment. A general loglinear analysis was completed to determine the 
relationship between teachers' educational attainment (degree plus coursework) and affiliation 
with school and group. No significant relationship was found, x 2 (28) = 0. 12,ps. Most teachers 
had a Masters degree or Masters plus coursework, n = 14 (52%) or a Bachelor degree, n = 8 
(30%). Three teachers (11%) had less than a bachelor degree and two teachers (7%) had a 
doctorate. 

Summary of Classroom and Teacher Differences. In sum, few differences existed 
between schools on classroom demographic or teacher experience variables, although there was 
considerable variability among individual classes and teachers. Two differences were noted 
within schools and classes or between schools. School 1 tended to have fewer students enrolled 



per class in vocational and integrated curricula classes. School 1 also tended to classify more 
students as honors students than School 2. 

Class types did differ on classroom demographics and teacher experience variables. Not 
surprisingly, academic classes tended to have more honors students and special education 
classes tended to have more special education students. Students with disabilities were more 
often mainstreamed in vocational and integrated classes than in academic classes. On average, 
integrated curricula classes had the highest rating for academic rigor. Vocational and integrated 
curricula classes tended to have higher ratings for technical rigor. Academic and integrated 
teachers were more likely to estimate that a greater percentage of their students were pursuing an 
academic path in high school. Vocational teachers as a group had more years experience for 
each teacher variable considered. 

Measures 

McDonnell and Ormseth (1989) noted that classroom practice indicator systems should 
address dimensions of content (e.g., curriculum area), depth (e.g., time on specific topics) and 
method of coverage (e.g., teacher style of presentation). Current curriculum restructuring efforts 
also support this multi-dimensional model (McDonnell, 1989). As suggested by Shavelson et al. 
(1991), and McDonnell et al. (1990), the selected measures in this study were based on 
theoretical models found in the literature (e.g., Adelman, 1989; Grubb et al., 1991; Plihal et al., 
1992) and encompassed aspects of content, depth, and method of coverage. The classroom 
observation system in this study measured five dimensions of curriculum and instruction based 
on a model of integrated curricula adapted from Plihal et al. (1992): (a) content blend, (b) 
vocational content, (c) instruction strategy, (d) student activity, and (e) instruction grouping. 
Within these dimensions, categories were defined to reflect a range of possible activities 
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discussed in the literature. These categories are defined and examples given in Appendix A. 
Examples were derived from the literature (e g., Reid & Tsuzuki, 1994) and the author's pilot 
classroom observations. 

The content blend dimension represented the degree of academic and occupational 
content integration (academic only, academic/reinforced occupational, fused 
academic/occupational, occupational/reinforced academic, and occupational only). The 
vocational content dimension identified the intended outcome of vocational-oriented instruction 
(none, general career awareness, general work behavior, general work skill, job-specific skill). 
The instruction strategy dimension described the teacher activity (monitor, lecture/review, lead 
discussion, modeling, coaching). The student activity dimension represented the type of student 
behavior demonstrated during an instructional activity (listen/take notes, drilled response, 
discussion, lab/applied - routine, lab/applied - novel). Instruction grouping identified whether 
the teacher worked with individuals, partners, small groups, or the whole class. Within the 
context of this study, the five dimensions and their categories were the school-based classroom- 
level variables used to demonstrate the construct of integrated curricula by differentiating four 
known groups of classes: (a) academic, (b) vocational, (c) integrated academic/occupational, and 
(d) special education academic. 

Instruments 

Data were collected using multiple instruments similar to those developed by Porter et al. 
(1993) to assess the enactment of math and science curricula at the secondaiy level. The 
specific measures in the instruments for this study reflected the dimensions and categories 
described above. Multiple instruments were used to demonstrate the degree to which differences 
on the measures were evident across data collection methods. 
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instruments included: (a) an Administrative Enrollment form for documenting course 
enrollment, student classification, and course type; (b) a Course Questionnaire to document 
teachers' initial estimates of time spent in curriculum and instruction areas represented by the 
measures, teachers' professional background, and teachers' verification of the course as a 
particular type (i.e., academic, vocational, integrated, special education); (c) Daily Logs, on 
which, teachers reported daily class activities and described each activity using coding 
categories representing the specific measures within each dimension; (d) Observer Logs, which 
were similar to the Daily Logs, but were completed by research staff, and provided additional 
contextual detail such as total length of class and number of students present; and (e) Weekly 
Logs that provided a vehicle for teachers to report anecdotal information regarding any 
difficulties encountered in using the coding system, special classroom activities not captured by 
the daily log, changes in the school's daily functioning that may have affected classroom 
activities, and any questions regarding the project. Also, a follow-up survey was used to examine 
teachers' post-study perceptions of the data collection process and initial outcomes of the study. 

Prior to formal data collection, the author tested pilot instruments in several classes and 
asked school teachers, administrators, and university professors familiar with classroom data 
collection to review the forms. Based on these informal observations and discussions, the 
instruments were modified to clarify ambiguous wording and to make recording data as simple 
and efficient as possible. Samples of the data collection instruments are included in Appendix B. 

Procedures 

Selection of Participants 

First, the author received approval to observe the schools from the local education 
agency research director’s office and participating schools' principals. Each principal identified 
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one or more assistant principals or curriculum specialists to act as administrative contacts within 
the schools. These school administrators identified courses that represented academic, 
vocational, integrated curricula, and special education classes and teachers that might be 
interested in participating in the study. The author approached each identified teacher to explain 
a general purpose of the study (to test an observation system in a variety of classes), observed 
each class, and invited the teacher to participate. Teachers were offered an honorarium ($100) if 
they participated in the project. 

If the teacher agreed to participate, the author requested the classroom demographic 
information from administrators. Requested data included class enrollment, number of 
males/females and student classification (e.g., low income). This information typically was 
supplied in the form of computer reports generated from the schools' administrative databases. 
The author and research assistants transferred this information to Administrative Enrollment 
forms. Each form was checked for accuracy by a second research assistant. 

Teacher Training 

Participating teachers received one to two hours of small group and individual training. 
During the training, the author reviewed each of the measures and definitions qf coding 
categories. Also, each teacher was asked to describe a recent class he or she had taught. These 
sample activities were discussed and coded to help teachers clarify definitions as well as practice 
coding. Each teacher was given a manual that contained an explanation of the general purpose of 
the project, the measures and definitions, additional examples of activities illustrating each 
measure, and instructions on how to complete the required documentation. The manuals also 
contained all the forms and logs needed by the teachers to complete the project (except the 
follow-up survey) and a calendar showing due dates for each form or log. A sample manual is 
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shown in Appendix C. Following the initial training, teachers were asked to review the manual 
and complete additional daily logs for practice. These were not collected by the author. Before 
beginning daily data collection, the author met with each teacher to answer questions about 
measures and definitions. 

Teacher-reported Data Collection 

Teachers were asked to submit the Course Questionnaire after completing the initial 
training and some practice daily logs, but before beginning actual in-class data collection. For a 
period of three weeks (fifteen days), teachers completed Daily Logs. Data collection occurred 
over a total of six consecutive weeks (three at each school) within a single semester. By 
coincidence, all teachers missed one day of data collection because of unplanned system-wide 
school closures. Thus, there were 14 possible Daily Log data collection days for each class. The 
teacher was instructed to complete the Daily Log as soon as possible following each selected 
class. Teachers could indicate on the Daily Logs that they could not identify an appropriate 
coding category within a dimension for a particular activity by making an “X” through the 
dimension coding box for that activity. At the end of each week, the teacher completed the 
Weekly Log. Teachers submitted logs on a daily or weekly basis to a centrally located mailbox 
in their schools. As logs were collected, the author examined them for completeness. If the logs 
were incomplete or missing (i.e., not submitted by the first day after the end of the week), the 
author contacted the teacher and requested the needed information. Missing data are discussed in 
the Data Analysis section. 

Observer Training 

Prior to in-class data collection, the research assistants received approximately ten hours 
of small group and individual training. Initial training addressed the general purpose of the 



project, the coding category definitions, and completion of the required forms. In addition, the 
research assistants practiced coding written narrative descriptions of multiple classroom 
scenarios. The narratives were developed by the author based on actual class activities observed 
while piloting the instruments. The research assistants achieved an average overall interobserver 
accuracy score of 82% (range 75% - 86%). Accuracy scores on dimensions ranged from 69% 
for Content Blend to 92% for Instruction Strategy. Accuracy was calculated as percent items 
scored correctly compared to a response sheet prepared by the author. Research assistants also 
observed at least two classes prior to the daily data collection period. During these practice 
observations, they completed the chronological narratives and observer logs. These were 
reviewed and discussed with the author. 

To promote interobserver agreement, the research assistants and author met weekly 
during the data collection period to review the measures and to conduct interobserver agreement 
tests using multiple video segments and a written narrative. Agreement was calculated as 
number of agreements divided by number of agreements plus number of disagreements. During 
this period, overall interobserver agreement on the videos averaged 80% (overall session range = 
55% to 100%; average dimension agreement range = 69% on Student Activity to 88% on 
Content Blend and Instruction Grouping), Overall agreement on the written narrative was 95% 
(average dimension agreement range = 89% on Content Blend to 100% on Student Activity and 
Instruction Grouping), 

Observer-reported Data Collection 

Research assistants conducted classroom observations in each class at least three times 
and as often as five times during the data collection period. Teachers were aware that they would 
be observed, but were not informed of the observation schedule. Observations were scheduled by 



randomly selecting five numbers corresponding to possible data collection days for each class, if 
possible, observations missed due to teacher or research assistant absences were rescheduled. A 
total of 1 12 observations were completed (approximately 30% of 378 total daily logs generated). 

To reduce teacher and student reactivity, research assistants were instructed to be as 
unobtrusive as possible. For instance, they were to choose a seat on the side or back of the 
classroom, but close enough to see the materials being used by the students. Also, the research 
assistants were instructed to have few, if any, interactions with the teachers and students. The 
research assistants were introduced to the teachers before the daily data collection period, and 
teachers were asked to briefly introduce the research assistants to the students in their class. 

If the teacher or students moved around the room during the class, the observers could 
move around also, as long as this did not disrupt the activity. The research assistants were 
permitted to look at materials given to students by the teacher or ask students at appropriate 
times about their assignment if the observer needed help to identify the content. If some students 
left the class to complete a course-related activity in another area of the school, the observer 
would follow them, make notes about their activity, and then return to the place where most of 
the students were working. 

During each observation the observers made a chronological narrative account of 
classroom activities as they occurred. They used a written guide to prompt them to document 
activities related to the study's dimensions and categories. The research assistants were 
instructed to make notes that would provide enough information to help them select and justify a 
code for each category of interest. The classroom observation notes were attached to the 
completed observation log. Both the chronological notes and Observation Logs were submitted 
on a daily or weekly basis to the author. 
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Interobserver Agreement 

Daily logs and observer logs were compared to determine agreement between teachers 
and research staff when selecting coding categories to describe activities. One hundred ninety- 
one (191) activities across 1 12 observations were analyzed. Common disagreements were 
identified by noting if more than 5% of the discrepancies within any single dimension occurred 
as a specific coding pair (e.g., teacher coded Content Blend dimension as “fused” and observer 
coded Content Blend as “occupational only”). These common disagreements were identified 
across all activities, by group and by school. 

Kappa values (Cohen, 1960) for interobserver agreement were moderate (0.56) to low 
(0.35), £< 0.001 (Hartmann & Wood, 1990). The Vocational Content dimension had the highest 
proportion of agreements, 71%, kappa = 0.56. There were no common disagreements identified. 
Student Activity had 61% agreement, kappa = 0.50. A common disagreement for coding within 
this dimension occurred when teachers selected “listen/take notes/test” and research assistants 
chose “drilled response” for the same activity. This disagreement occurred for 8% (n.= 16) of 
the activities. More than half (n = 9) of these disagreements occurred with academic teachers. 
The Instruction Grouping dimension had agreement of 65%, kappa = 0.44. A common 
disagreement occurred as a result of teachers coding an activity as “individual” and research 
assistants coding the activity as “whole class”, n= 18 ( 9%). Half of these disagreements 
occurred with special education teachers. 

Proportion of agreement for the Content Blend dimension was 52%, kappa = 0.35. 
Common disagreements included teachers coding an activity as “fused”, but research assistants 
coding the same activity as “occupational only”, n = 20 (1 1%). Integrated curricula teachers 
accounted for 65% (n= 13) of these disagreements, and vocational teachers accounted for the 



remaining disagreements of this type (n = 7). Another common disagreement occurred when 
teachers selected “occupational, reinforce academic” and research assistants chose 
“occupational only”, n = 20 (11%). Ninety percent of these disagreements occurred with School 
1 teachers; sixty percent were attributable to integrated curricula teachers. 

The Instruction Strategy dimension also had a low proportion of agreement, 53%, kappa 
= 0.35. Common disagreements included teachers coding an activity as “lecture/review” and 
research assistants coding the same activity as “monitor”. This disagreement occurred for 5% (n 
= 10) of the activities. Half of these disagreements occurred with special education teachers. 
Nine percent (n = 18) of the common disagreements in this dimension were attributable to 
teachers coding an activity as “coaching” and research assistants coding it as “monitor”. 
Approximately 72% (n= 13) of these disagreements occurred with School 2 teachers. 

Data Analysis 

Discrimination. Convergence. Reliability, and Relevance 

The primary focus of data analysis was the construct validity of the measures. Construct 
validity was first examined using a "known group" strategy with the intent of demonstrating 
hypothesized differences between groups (Morris et al., 1987). Kruskal-Wallis nonparametric 
one-way analysis of variance tests (Kruskal & Wallis, 1952; Siegel & Castellan, 1988) were used 
to determine if significant differences (at the .05 level) existed between class types' (academic, 
vocational, integrated, special education) mean ranks on summary variables representing the five 
dimensions of curriculum and instruction. In cases where more than 25% of the classes tied for 
ranks, which may have inflated Kruskal-Wallis values, uncorrected scores were computed to 
verify significance. When significant values were found, follow-up tests were conducted to 
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identify the source of the differences using the formula recommended by Siegel and Castellan 
(1988). 

Construct validity was further examined in a second set of analyses using Kendall’s W, a 
nonparametric test of concordance, to determine the degree to which the different instruments 
yielded similar rankings of the groups on the summary measures. Additionally, as described 
above, reliability of the instruments and measures was tested with Cohen’s kappa (Cohen, 1960), 
which corrects for chance agreement. Finally, the content and social relevance of the dimensions 
and coding categories were explored using descriptive statistics from the follow-up survey and 
anecdotal information compiled from teacher-completed Daily and Weekly Logs, incidental 
conversations with teachers, and informal de-briefing interviews conducted with observers. 
Summary Variables 

All data from the original teacher- or observer-completed logs were entered into 
spreadsheets by research staff. Data entries were checked for accuracy by a second research 
assistant. The summary variables were constructed as follows to represent the features of 
integrated curricula described in the reviewed literature. General Vocational Content was the 
sum of time spent on (a) general career awareness, (b) general work behaviors, and (c) general 
work skills. Reinforced/Fused Content Blend was the sum of time spent on activities that were 
(a) primarily academic, but reinforced vocational content, (b) primarily vocational, but 
reinforced academics, and (c) fused academic and vocational. Coaching as an instruction 
strategy was represented only by time spent on coaching, Lab/Applied Work included student 
activities that were described as (a) lab/applied, routine and (b) lab/applied, novel. Cooperative 
Grouping consisted of time spent as (a) small groups and (b) partners. Theoretically, classes that 
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integrated academic and occupational curricula would tend to have higher ranks on these 
summary variables. 

Separate analyses were conducted for the Course Questionnaire, the Daily Logs, and the 
Observer Logs. For the Course Questionnaire, class ranks were based on constructing the 
summary variables from the percent time spent on activities as noted by the teacher. For the 
Daily Logs, the ranks for each class were based on the total percent class time spent on the 
summary variables across all (14) data collection days. For the Observation Logs, class ranks 
were based on the average percent time spent on the summary variables across observations (3 - 
5 per class). For descriptive purposes, the Daily Logs total percent time was converted to average 
percent time by dividing the total by 1400 (based on 14 data collection days). 

Missing Data 

Missing at>d suspect data on Daily Logs that could not be addressed by the teacher were 
handled in one of five ways at the end of data collection. If a teacher was absent for a data 
collection day, the teacher's average time spent on each category was used for that day. If a 
teacher did not select a category in a particular dimension, the teacher's averages for time spent 
on the categories in that dimension were used. However, if a teacher explicitly indicated that 
none of the dimension categories were appropriate for a particular activity, then a 0 (no time) 
was assigned to each category in that dimension for that activity. If a teacher circled two 
categories within a dimension for a single activity and did not indicate that one was his/her first 
choice, then time spent on that activity was divided equally between the two categories. Finally, 
teachers occasionally chose incongruous categories across dimensions. For example, a teacher 
selected Ha Vocational Content as the category that best described the Vocational Content of an 
activity, and then selected Academic, Reinforced Occupational in the Content Blend dimension. 
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suggesting that some vocational content had been addressed. In these cases, the teacher's 
averages were substituted in the two problematic dimensions for that activity. 

The number of missing logs due to absences and the number of complete logs (all 
activities categorized in each dimension by the teacher) were analyzed using a two factor 
analysis of variance to explore whether missing data might create unintended differences among 
groups. The two factors were school (School 1, School 2) and group (academic, vocational, 
integrated, special education). The analysis on missing data due to absences (M = 1.78. sd = 

1 .09) indicated that there were no significant interactions for school and group, F (3, 19) = 0. 1 5, 
ns, and there were no main effects for school, F (1, 19) = 0.1 1, ns, or group, E.(3, 19) = 1.51, as. 
The number of complete logs (M= 1 1, S&= 3.63) were similarly analyzed. The analysis 
indicated that there were no significant interactions for school and group, E_(3, 19) = 0.33, ns, 
and there were no main effects for school, E(l, 19) = 1.84, ns, or group, E (3, 19) = 0.31, ns. 
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CHAPTER III 



RESULTS 

Four sets of analyses were completed to examine the construct validity of the measures 
of integrated curricula: (a) discriminating between groups, (b) convergence across multiple 
instruments, (c) reliability of measures and instruments and (d) content and social relevance. 
Results for discrimination, convergence, and relevance are presented below. Results for 
reliability analyses (interobserver agreement for research assistants and teachers) are found in 
the preceding Methods section. 

Discriminating Between Groups 

Table 3 displays the mean ranks for summary variables by class type and results of 
Kruskal-Wallis analysis of variance tests for all instruments. Table 4 shows the average time 
engaged on summary variables within each dimension by class type on all instruments. 
Additional tables in this section display the average percent time engaged on specific categories 
within each dimension by class type and instrument. Findings for each of the five integrated 
curricula dimensions are presented. 

Vocational Content 

According to the course questionnaire and the daily logs, there were significant 
differences between class types for time spent on general vocational content. Follow up tests for 
both the course questionnaire and daily log results showed that integrated classes had 
significantly higher mean ranks than academic classes on general vocational content, | R , - R 3 1 = 
13.24 and 12.36 respectively; critical value = 1 1.66, z_= 2.638, p.< 0.05. Table 4 shows that on 
all instruments integrated classes, on the average, spent more time covering general 
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Table 3. Mean Ranks on Summary Variables 



Class Types 



Summary Variable 
Instrument 


Academic 
n = 6 


Vocational 
n = 7 


Integrated 
2 = 7 


Special 
Education 
2 = 7 






R 


R 


£ 


R 


XX H 


General Vocational 
Content 


Course Questionnaire 


6.83 


16.36 


20.07 


11.71 


10.16*° 


Daily Logs 


7.50 


15.14 


19.86 


12.57 


8.30* 


Observer Logs 


7.50 


13.71 


16.86 


17.00 


6.90 


Reinforce/Fused 
Content Blend 


Course Questionnaire 


10.75 


16.57 


18.00 


10.21 


5.16 


Daily Logs 


6.00 


19.57 


19.43 


9.86 


14.82* 


Observer Logs 


7.50 


19.14 


17.29 


11.14 


9.05* 8 


Coaching Instruction 
Strategy 


Course Questionnaire 


9.42 


20.07 


14.43 


11.43 


6.92 


Daily Logs 


11.00 


15.43 


15.50 


13.64 


1.35 


Observer Logs 


14.50 


11.79 


16.57 


13.21 


2.11 


Lab/Applied Student 
Activity 


Course Questionnaire 


8.92 


19.50 


15.86 


8.33 


9.86* 8 


Daily Logs 


5.00 


22.43 


15.57 


11.71 


16.47* 


Observer Logs 


5.50 


19.57 


18.79 


10.93 


13.92*“ 


Cooperative Instruction 
Grouping 


Course Questionnaire 


10.50 


18.21 


19.43 


7.36 


1 1.29*° 


Daily Logs 


11.75 


15.79 


19.86 


8.29 


8.30* 


Observer Logs 


14.00 


16.07 


16.79 


9.14 


4.17 



8 Uncorrected value 
* Significant at £ < 0.05 
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Table 4. Average Percent Time Engaged on Summary Variables 



Class Types 



Summary Variable 
Instrument 


Academic 
n = 6 


Vocational 
n = 7 


Integrated 
n = 7 


Special 
Education 
n = 7 




M (SD) 


M (SD) 


M (SD) 


M 


(SD) 


General Vocational 
Content 


Course Questionnaire 


20.33 (16.81) 


57.86 (25.64) 


72.14 (22.52) 


40.71 


(32.33) 


Daily Logs 


9.46 (20.14) 


35.44 (35.34) 


60.79 (36.28) 


24.52 


(21.66) 


Observer Logs 


0.00 (0.00) 


24.02 (40.62) 


35.58 (34.16) 


20.04 


(19.87) 


Reinforced/Fused 
Content Blend 


Course Questionnaire 


37.83 (48.29) 


74.71 (20.26) 


84.14 (11.48) 


35.00 


(37.52) 


Daily Logs 


9.46 (20.14) 


73.04 (21.76) 


69.31 (37.15) 


20.27 


(20.76) 


Observer Logs 


0.00 (0.00) 


26.54 (16.88) 


31.98 (32.78) 


3.32 


(4.39) 


Coaching Instruction 
Strategy 


Course Questionnaire 


16.67(11.69) 


52.86 (26.28) 


30.00 (23.63) 


22.14 


(13.80) 


Daily Logs 


19.36 (20.38) 


30.56 (27.88) 


29.37 (15.57) 


27.40 


(16.39) 


Observer Logs 


6.88 (11.23) 


2.38 (6.30) 


11.79 (15.59) 


3.81 


(6.79) 


Lab/Applied Student 
Activity 


Course Questionnaire 


21.67 (32.35) 


62.86 (18.23) 


52.14 (14.10) 


18.67 


(24.55) 


Daily Logs 


11.30 (11.73) 


70.02 (17.03) 


45.13 (25.01) 


29.50 


(23.87) 


Observer Logs 


2.50 (6.12) 


55.17 (22.82) 


49.44 (28.44) 


21.28 


(34.03) 


Cooperative Instruction 
Grouping 


Course Questionnaire 


25.00(15.17) 


50.86 (25.83) 


55.71 (24.91) 


16.71 


(8.98) 


Daily Logs 


15.96(15.11) 


30.68 (27.80) 


40.48 (27.28) 


7.84 


(7.09) 


Observer Logs 


14.50 (9.37) 


30.71 (31.02) 


30.27 (30.54) 


5.34 


(7.85) 
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vocational content than all other types of classes. Vocational classes had the second highest 
average percent time on general vocational content. Next highest were special education 
academic classes, and on average, academic classes spent the least amount of time on general 
vocational content. Table 5 shows the average percent time the class types spent on each 
category within the Vocational Content dimension. 

Content Blend 

On the daily logs and observer logs, significant group differences were apparent for the 
amount of time classes experienced reinforced or fused content blend. Follow up tests on the 
daily logs results indicated that integrated classes had significantly higher mean ranks than 
academic classes, | R, - R 3 1 = 13.43, critical value = 1 1.66, z = 2.638, p < 0.05. On both the daily 
logs and observer logs, vocational classes also ranked higher than academic classes, | R, - R 2 1 = 
13.57 and 1 1.64 respectively, critical value = 11.66, £ = 2.638, p< 0.05. It should be noted that 
post hoc test results for the observation logs approached but did not reach the critical value 
necessary for significance. Table 4 shows that integrated classes and vocational classes spent 
more time on average reinforcing or fusing academic and vocational content than academic or 
special education academic classes. This pattern was evident across all instruments. Table 6 
shows the average time spent by classes for each category within the Content Blend dimension. 
Instruction Strategy 

There were no significant differences for coaching as an instruction strategy on any of 
the instruments. Table 4 shows that across all instruments, on the average, integrated curricula 
and vocational class teachers spent more time coaching than academic or special education 
academic class teachers. According to the observer logs, teachers in all class types spent 
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Table 5. Vocational Content Categories: Average Percent Time Engaged 



Class Types 



Category 

Instrument 


Academic 
n = 6 


Vocational 
n = 7 


Integrated 
n = 7 


Special 
Education 
n = 7 




M 


(SD) 


M (SD) 


M (SD) 


M 


(SD) 


No Vocational Content 


Course Questionnaire 


79.67 


(16.81) 


0.71 (1.89) 


11.43 (26.10) 


52.14 


(40.19) 


Daily Logs 


90.54 


(20.14) 


0.76 (1.36) 


25.09 (39.10) 


62.26 


(38.48) 


Observer Logs 


100.00 


(0.00) 


1.57 (4.16) 


41.76 (43.68) 


76.63 


(26.07) 


General Career 
Awareness 


Course Questionnaire 


7.00 


(7.21) 


8.57 (6.27) 


27.86 (19.58) 


12.86 


(10.75) 


Daily Logs 


0.75 


(1.84) 


10.36 (10.78) 


36.28 (32.81) 


8.77 


(16.90) 


Observer Logs 


0.00 


(0.00) 


0.00 (0.00) 


6.99 (10.13) 


8.32 


(9.43) 


General Work Behavior 


Course Questionnaire 


7.50 


(7.58) 


13.57 (10.69) 


13.57 (9.00) 


16.43 


(15.74) 


Daily Logs 


6.11 


(13.97) 


3.58 (7.43) 


3.84 (7.05) 


4.442 


(6.17) 


Observer Logs 


0.00 


(0.00) 


0.00 (0.00) 


0.00 (0.00) 


0.00 


(0.00) 


General Work Skill 


Course Questionnaire 


5.83 


(12.01) 


35.71 (22.63) 


30.71 (19.46) 


11.43 


(10.69) 


Daily Logs 


2.61 


(6.39) 


21.29 (34.22) 


20.67 (18.93) 


11.34 


(15.11) 


Observer Logs 


0.00 


(0.00) 


24.02 (40.62) 


28.59 (29.61) 


11.71 


(18.02) 


Joh^specific Skill 


Course Questionnaire 


0.00 


(0.00) 


41.43 (25.28) 


16.43 (14.06) 


7.14 


(9.51) 


Daily Log 


0.00 


(0.00) 


64.00 (35.14) 


14.12 (23.99) 


13.22 


(21.30) 


Observer Logs 


0.00 


(0.00) 


74.41 (39.74) 


19.09 (36.86) 


3.33 


(8.82) 
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Table 6. Content Blend Categories: Average Percent Time Engaged 



Class Types 



Category Academic 



Instrument 


A = 6 




M (SD) 


Academic Only 




Course Questionnaire 


61.33 (47.55) 


Daily Logs 


90.54 (20.14) 


Observer Logs 


100.00 (0.00) 


Academic. Reinforce 




Occupational 




Course Questionnaire 


19.17 (39.80) 


Daily Logs 


3.00 (5.45) 


Observer Logs 


0.00 (0.00) 


Fused 




Course Questionnaire 


17.00 (40.67) 


Daily Logs 


6.46 (14.84) 


Observer Logs 


0.00 (0.00) 


Occupational 




Reinforce Academic 




Course Questionnaire 


1.67 (4.08) 


Daily Logs 


0.00 (0.00) 


Observer Logs 


0.00 (0.00) 


Occupational Onlv 




Course Questionnaire 


0.00 (0.00) 


Daily Log 


0.00 (0.00) 


Observer Log 


0.00 (0.00) 



Special 

Vocational Integrated Education 



n = 


= 7 


n = 


= 7 


n = 


= 7 


M 


CSD) 


M 


(SO) 


M 


(3D) 


2.86 


(3.93) 


11.43 


(11.07) 


40.00 


(40.10) 


0.76 


(1.36) 


25.30 


(38.98) 


53.97 


(41.18) 


1.57 


(4.16) 


41.68 


(43.76) 


64.11 


(31.99) 


" 7.14 


(9.51) 


24.71 


(17.14) 


19.29 


(21.69) 


14.81 


(12.21) 


17.68 


(21.64) 


7.06 


(7.73) 


0.00 


(0.00) 


14.86 


(24.05) 


3.32 


(4.39) 


29.29 


(32.33) 


34.71 


(15.02) 


4.43 


(9.25) 


26.36 


(32.59) 


27.96 


(30.93) 


2.12 


(3.65) 


0.00 


(0.00) 


9.52 


(25.20) 


0.00 


(0.00) 



38.29 


(23.46) 


24.71 


(19.21) 


11.29 


(12.87) 


31.43 


(30.36) 


23.66 


(23.24) 


11.08 


(14.83) 


26.54 


(16.88) 


7.60 


(13.01) 


0.00 


(0.00) 


22.43 


(19.23) 


2.86 


(3.93) 


7.14 


(12.20) 


26.64 


(21.95) 


5.40 


(7.41) 


16.19 


(21.68) 


71.89 


(14.32) 


26.34 


(38.82) 


14.95 


(18.13) 



relatively small amounts of time using coaching as an instruction strategy. Table 7 shows the 
average percent time-spent by teachers for each category of Instruction Strategy. 

Student Activity 

On all instruments, class types differed significantly for time engaged in lab/applied 
student activity. On the observer logs, integrated curricula classes had significantly higher mean 
ranks than academic classes, | R, - R 3 1 = 13.29, critical value = 1 1 .66, z = 2.638, p < 0.05. 
Vocational classes ranked significantly higher than academic classes on both the observer logs 
and daily logs, | R, - R, 2 1 = 17.43 and 14.07 respectively, critical value = 1 1 .66, z = 2.638, p < 
0.05. On the course questionnaire, vocational classes had higher mean ranks than special 
education classes, | R 2 - R 4 1 = 11.17, critical value = 10.58, z_= 2.394, p < 0.05. Table 4 shows 
that on all instruments vocational class students spent more time on average doing lab/applied 
activities than all other class types. Integrated class students were next highest on all instruments 
in terms of student time on applied activities. Academic class students spent the least time doing 
applied activities, except as reported by teachers on the course questionnaire. Table 8 provides 
the average percent time that students spent on each Student Activity category. 

Instruction Grouping 

Class types differed significantly for time spent on cooperative instruction grouping, 
according to the course questionnaire and the daily logs. In both cases, integrated classes had 
significantly higher mean ranks for cooperative instruction grouping than special education 
classes, | R 3 - R, | = 12.07 and 1 1,57 respectively, critical value = 1 1 .20, z = 2.638, p < 0.05. As 
shown in Table 4, students in integrated curricula classes and vocational classes spent more time 
on average in cooperative instructional groupings than students in academic or special education 
academic classes. Table 9 shows the average percent time that students spent in each category of 
Instruction Grouping. 



Table 7. Instruction Strategy Categories: Average Percent Time Engaged 



Class Types 



Category 

Instrument 


Academic 
n = 6 


Vocational 
n = 7 


Integrated 
n = 7 


Special 
Education 
n = 7 




M (30) 


m cs m 


M (3D) 


M (3D) 


Monitor 


Course Questionnaire 


25.00 (24.50) 


11.43 (17.49) 


29.29 (24.57) 


10.00 (10.41) 


Daily Logs 


39.23 (24.51) 


27.73 (17.65) 


42.98 (23.78) 


20.11 (20.98) 


Observer Logs 


39.53 (9.82) 


60.71 (19.02) 


48.11 (28.94) 


40.88 (29.85) 


Lecture/Review 


Course Questionnaire 


41.67 (19.41) 


12.86 (6.36) 


16.43 (10.29) 


30.29 (16.95) 


Daily Logs 


29.70 (12.84) 


13.28 (6.76) 


13.29 (12.56) 


27.41 (11.71) 


Observer Logs 


34.07 (13.89) 


15.87 (15.82) 


15.96 (19.89) 


34.19 (22.83) 


Lead Discussion 


Course Questionnaire 


12.50 (14.41) 


8.57 (6.27) 


10.00 (7.64) 


19.29 (6.73) 


Daily Logs 


8.33 (7.18) 


7.57 (9.11) 


8.98 (14.59) 


12.91 (11.96) 


Observer Logs 


3.38 (5.40) 


0.00 (0.00) 


8.57 (14.92) 


9.67 (18.31) 


Modeling 


Course Questionnaire 


5.83 (4.92) 


14.29 (16.44) 


17.86 (11.85) 


18.29 (14.16) 


Daily Logs 


3.08 (4.62) 


20.89 (22.25) 


5.39 (3.22) 


12.16 (9.31) 


Observer Logs 


16.15 (18.78) 


21.04 (20.43) 


15.57 (18.37) 


11.45 (11.57) 


Coaching 


Course Questionnaire 


16.67 (11.69) 


52.86 (26.28) 


30.00 (23.63) 


22.14 (13.80) 


Daily Log 


19.36 (20.38) 


30.52 (27.92) 


29.37 (15.57) 


27.40 (16.39) 


Observer Log 


6.88 (11.23) 


2.38 (6.30) 


11.79 (15.59) 


3.81 (6.79) 
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Table 8. Student Activity Categories: Average Percent Time Engaged 



Class Types 

Special 

Category Academic Vocational Integrated Education 

Instrument n = 6 n = 7 n = 7 n = 7 



Listen/Take Notes/Test 
Course Questionnaire 
Daily Logs 
Observer Logs 
Drilled Response 
Course Questionnaire 
Daily Logs 
Observer Logs 
Discussion/Discoverv 
Course Questionnaire 
Daily Logs 
Observer Logs 
Lab/Applied. Routine 
Course Questionnaire 
Daily Logs 
Observer Logs 
Lab/Applied. Novel 
Course Questionnaire 
Daily Log 
Observer Log 



M 


(3D) 


M 


(3D) 


36.67 


(25.23) 


17.14 


(7.56) 


51.56 


(20.78) 


18.93 


(10.59) 


38.60 


(22.08) 


19.05 


(17.10) 


20.00 


(15.17) 


10.71 


(8.87) 


28.81 


(20.03) 


7.43 


(10.33) 


52.48 


(25.05) 


25.79 


(15.54) 


21.67 


(19.92) 


9.29 


(7.87) 


7.94 


(6.91) 


4.25 


(6.44) 


6.42 


(5.20) 


0.00 


(0.00) 


19.50 


(28.65) 


51.43 


(21.55) 


11.30 


(11.73) 


64.78 


(20.23) 


2.50 


(6.12) 


52.79 


(22.28) 


2.17 


(3.92) 


11.43 


(13.76) 


0.00 


(0.00) 


4.62 


(7.89) 


0.00 


(0.00) 


2.38 


(6.30) 



M 


(3D) 


M 


(3D) 


15.00 


(9.13) 


17.17 


(22.28) 


30.52 


(20.57) 


18.77 


(24.70) 


20.49 


(21.33) 


21.73 


(19.49) 


10.00 


(9.57) 


36.67 


(20.90) 


9.58 


(7.98) 


31.42 


(21.51) 


20.08 


(17.40) 


47.01 


(27.38) 


20.00 


(14.34) 


19.17 


(14.63) 


14.76 


(18.45) 


19.88 


(12.26) 


10.00 


(14.43) 


9.99 


(11.78) 


34.29 


(18.80) 


9.33 


(12.28) 


33.98 


(17.46) 


18.71 


(20.33) 


36.21 


(25.98) 


9.14 


(12.75) 


17.86 


(19.12) 


9.33 


(12.28) 


11.15 


(14.15) 


10.79 


(14.59) 


13.23 


(17.19) 


12.14 


(24.99) 



Table 9. Instruction Grouping Categories: Average Percent Time Engaged 



Class Types 



Cateeorv 

Instrument 


Academic 
n = 6 


Vocational 
n = 7 


Integrated 
n = 7 


Special 
Education 
n = 7 




M (SO 


M (SO 


M (SO 


M (SO 


Whole Class 


Course Questionnaire 


60.83 (24.99) 


41.14 (24.20) 


31.43 (25.94) 


70.00 (18.03) 


Daily Logs 


77.82 (17.07) 


52.39 (32.81) 


49.15 (25.46) 


64.92 (16.51) 


Observer Logs 


81.33 (9.72) 


51.16 (34.27) 


50.10 (41.61) 


81.14 (10.60) 


Small GrouDS 


Course Questionnaire 


17.50 (9.87) 


30.86 (23.60) 


34.29 (21.88) 


7.71 (10.72) 


Daily Logs 


11.23 (13.19) 


26.43 (27.98) 


23.24 (15.07) 


7.14 (6.57) 


Observer Logs 


9.92 (9.27) 


26.00 (28.77) 


25.84 (31.72) 


5.35 (7.85) 


Partners 


Course Questionnaire 


7.50 (9.87) 


20.00 (22.36) 


21.43 (27.50) 


9.00 (10.18) 


Daily Logs 


4.74 (8.64) 


3.54 (7.99) 


17.25 (34.43) 


0.69 (1.36) 


Observer Logs 


4.58 (7.14) 


4.71 (8.66) 


4.43 (8.44) 


0.00 (0.00) 


Individuals 


Course Questionnaire 


14.17 (18.55) 


8.00 (5.45) 


10.00 (6.46) 


13.29 (15.56) 


Daily Logs 


6.21 (4.96) 


17.64 (25.92) 


10.37 (5.20) 


27.24 (17.83) 


Observer Logs 


4.17 (5.85) 


18.13 (22.62) 


19.63 (26.53) 


13.51 (8.12) 
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Convergence Across Instruments 



There was a high degree of concordance across instruments for groups' mean ranks on all 
summary variables except coaching instruction strategy. Mean ranks are displayed in Table 3. 
Kendall's iV values are displayed in Table 10. One teacher (special education) did not provide 
estimates on the course questionnaire for time spent on Student Activity categories. Therefore, 
that teacher was not included in the concordance test for Student Activity, reducing the number 
of possible data points to 26 for that particular test. All classes were included in the concordance 
tests for the other dimensions. 



Table 10. Kendall's W (Coefficient of Concordance) for Mean Ranks on Instruments 



Summary Variable 


_ _ x 2a 




12 


General Vocational Content 


61.58 


0.79 


<0.001 


Reinforced/Fused Content Blend 


50.69 


0.65 


0.003 


Coaching Instruction Strategy 


26.49 


0.34 


0.44 


Lab/Applied Student Activity 


59.75 


0.80 


<0.001 


Cooperative Instruction Grouping 


52.72 


0.68 


0.001 



a df= 26, except Student Activity dimension df = 25 



Content and Social Relevance 

Eighteen (67%) of the 27 teachers in the sample completed the follow-up survey. School 
1 and School 2 were represented by equal numbers of respondents. Respondents included all six 
of the final sample academic teachers, five of seven integrated curricula teachers, four of seven 
special education teachers, and three of seven vocational teachers. 
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Data Collection 



The first section of the follow-up survey asked teachers about their perceptions of the 
classroom data collection process. Responses were structured as Likert-type scales with four 
points. Seventy-eight percent (n = 14) of the respondents reported that completing the course 
questionnaire took twenty minutes or less. Half of these were able to complete the course 
questionnaire in 10 minutes or less. Four teachers took 21-30 minutes to complete the course 
questionnaire. All responding teachers reported that the daily log usually could be completed in 
10 minutes or less. Forty-four percent were able to complete it in five minutes or less. Eighty- 
nine percent (n = 16) of the respondents noted that the format of the daily logs made 
documenting classroom activities fairly or very easy. Teacher's written anecdotal comments also 
suggested that the daily logs were simple to use. Two noted that documenting classroom 
activities was a little difficult. Fourteen (78%) of the teachers reported they felt fairly or very 
confident about choosing among the codes in each dimension on the daily log. The remaining 
four reported being a little unsure. Eighty-nine percent (q = 16) of the teachers said that the 
observers (research assistants) were not at all disruptive to the classroom. Two said the observers 
were a little disruptive. Fifteen (83%) of the teachers said that observer presence did not at all 
influence how they documented and coded activities on the daily logs. Three reported that they 
were influenced a little. 

Results 

The second section of the survey asked each teacher to examine summary information 
for his or her class and the four class types. The summary information included the daily log data 
only, expressed as percent class time engaged within each category and dimension. All teachers 
believed that the summary daily log information for their classes represented the activities that 



occurred in their classes during the project time fairly (67%) or very (33%) well. Ninety-four 
percent (n = 17) of the teachers also felt the summary information reflected the types of 
activities that occur in their class during an entire semester. One teacher suggested that the 
summary information reflected this only a little. 

Teachers also were presented with brief descriptions of the differences (statistical 
significance and directional differences of means) between class types based on daily log data. 
They were asked to comment on whether they believed the results accurately represented 
important differences between class types. Teachers could select yes, no or not sure. Space was 
provided for additional written comments. Teachers typically reported that they believed the 
results did represent important differences (ranging from 94% for the Content Blend dimension 
to 67% for Vocational Content and Instructional Grouping dimensions). Eighty-three percent of 
the teachers also supported the statement that special education classes were more likely than 
other types of classes to spend time on content other than academic or vocational (e.g., personal 
life skills). 

Utili ty 

Finally, teachers were asked to select from a list of possible ways in which the summary 
information about classroom activities might be useful. Teachers responses are presented in 
Table 1 1 . A recurring theme in teachers' written comments on the follow-up survey was the idea 
that keeping the daily logs and seeing summary information was useful for prompting teacher 
reflection on instruction. For example, one teacher noted that she gained insight into how much 
time she spent lecturing as opposed to using other instruction strategies. Another noted that she 
was made aware of how much time and about what other teachers teach, and this was helpful to 
her. 
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Tabje 1 1 . Teacher Support for Uses of Summary Information 



% (N = 18) 

Supporting 


Use 


83 


Describing some aspects of how I use classtime to other teachers 


83 


Describing some aspects of how I use classtime to school administrators 


83 


Describing some curricular and instructional differences between types 
of classes in my school 


72 


Monitoring some aspects of how I use classtime to inform my personal 
professional development 


67 


Monitoring some aspects of how curricula and instruction are 
implemented in my school 


56 


Monitoring some aspects of how I use classtime to inform a school-wide 
assessment of classroom activities 


0 


Other 


0 


None of the above 



Some teachers suggested in their written comments on the weekly logs and follow-up 
survey that some class activities may not have been well represented by the daily log data 
because (a) certain activities occur only at certain times in the semester, (b) choosing between 
codes was difficult for some activities, and (c) some activities focused on personal life skills, 
which were difficult to code on academic and vocational dimensions. 

Observers also noted this last problem when trying to code such activities as discussion 
about “getting an apartment”. Anecdotally, observers suggested that “coaching” became (for 
teachers) a “catch-all” description for a variety of instruction strategies, and the observers 
reported seeing little “real” coaching. Similarly, the observers reported seeing few examples of 
fused Content Blend, but they believed teachers expected students to recognize the implicit 
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academic or vocational value of activities. Several questions presented by teachers to the author 
at different times throughout training and data collection lent support to this belief. For example, 
some vocational teachers suggested that any school work that involved reading was implicitly 
academic even if the teacher didn't explicitly teach or evaluate reading skills or stress the 
academic relevance of reading to students. Likewise, some academic teachers suggested that all 
reading activities were inherently vocational because students would more than likely need to 
read on the job. 

Other coding problems identified anecdotally by observers related to the issue of 
embedding “high level” work in the Content Blend dimension categories' definitions. For 
example, in vocational classes and in the special education classes, the academic work 
expectations for some activities appeared low to observers although they believed that teachers 
explicitly blended academic and vocational work. Also, when coding some special education 
class activities, observers had difficulty coding activities that teachers called discussion. To the 
observers, these activities often appeared more like teacher-directed drilled response. 
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CHAPTER IV 



DISCUSSION 



The purpose of this study was to explore the validity of a set of observable measures 



representing integrated academic/occupational curricula in high school classrooms. These 
measures incorporated features of curriculum and instruction that, in recent school-to-work 
transition literature, have been associated with the construct of integrated academic/occupational 
curricula. Of particular interest was whether the set of measures would reflect significant 
differences among class types, including classes serving students with disabilities, and whether 
these differences could be observed across different types of instruments. Specifically, it was 
theorized that integrated curricula classes would have higher mean ranks on the measures than 
would other types of classes. To further examine validity, secondary objectives of the study were 
to determine (a) the reliability of the measures and instruments for describing integrated 
curricula and (b) the relevance of the resulting information. 



The findings provide support for the construct validity of the proposed measures of 
integrated academic/occupational curricula. The measures did differentiate between integrated 
or vocational classes and traditional academic or special education academic classes. The 
measures were less sensitive to differences between integrated and vocational classes. These 
results were demonstrated across multiple instruments. Findings of differences between groups 
and concordance across instruments are summarized below as qualitative differences, significant 
differences, concordance, and failure to discriminate. 

Qualitative differences. A national assessment of vocational education found that current 
levels of classroom academic/occupational integration were modest or minimal (Boesel et al.. 
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1994). Only 54% of vocational teachers reported spending at least 10% of classtime on any 
academic content and that content was most often at a basic skill level. Only 18% of academic 
teachers in the national assessment reported spending at least 10% of classtime on vocational 
content. This was most likely to occur in math classes. The present study also suggests that 
academic teachers spend little time (20% of classtime or less) on vocational topics of any type. 
Vocational teachers were more likely than academic teachers to report blending academic with 
vocational content. However, the extent to which this actually occurred is not clear and is 
discussed further below. 

Based on findings of average percent class time engaged, all instruments showed that 
teachers and students in integrated classes spent more time on all summary variables — general 
vocational content, reinforced/fused content blend, coaching instruction strategy, lab/applied 
student activity, and cooperative instruction grouping - than did academic or special education 
academic classes. Teachers and students in vocational classes also spent more time on these 
summary variables than academic and special education classes, except in the area of coaching 
as reported by observer logs. 

A similar pattern was evident in the mean ranks of classes on the summary variables. 
Integrated classes had higher mean ranks than academic or special education academic classes 
on all summary variables across all instruments, except that special education classes had a 
slightly higher mean rank for general vocational content as reported on observer logs. Vocational 
classes also had higher mean ranks than academic and special education classes on all summary 
variables across all instruments, except that vocational classes had the lowest mean rank as 
reported by observer logs for coaching. Generally, integrated academic/occupational classes and 




67 



vocational classes had the highest mean ranks on summary variables, and these two types of 
classes appeared most alike in percentages of classtime engaged on the summary variables. 

Significant differences. Tests of significance based on mean ranks typically demonstrated 
that vocational or integrated classes could be differentiated from academic or special education 
classes, except in the area of coaching as instruction strategy. Mediating these results is the fact 
that teacher-research assistant interobserver agreement was moderate to low, while interobserver 
agreement among observers was moderate to high. This suggests that the observer logs provide 
more reliable estimates of observable classroom activity. In fact, findings of significant 
differences between class types did vary by instrument type. 

Teachers in integrated curricula classes spent significantly more time covering general 
vocational content than those in academic classes, according to the course questionnaire and 
daily logs. Teachers in both integrated and vocational classes spent significantly more time 
providing reinforced/fused content blend than those in academic classes, according to the daily 
logs. There were no significant differences between class types on any instrument for teachers' 
use of coaching instruction strategy . According to the observer logs, students in both integrated 
and vocational classes spent significantly more time on lab/applied student activity than those in 
academic classes. The course questionnaire and daily logs also showed this difference between 
vocational and academic classes, as well as between vocational and special education classes. 
Finally, students in integrated classes spent significantly more time on cooperative instruction 
grouping than those in special education classes, according to the course questionnaire and daily 
logs. 

Concordance. Although findings of significance varied by instrument type, there was a 
strong concordance of rankings among the course questionnaire, daily logs and observer logs on 
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all summary variables except coaching In other words, classes that received higher rankings on 
one instrument also were more likely to receive higher rankings on other instruments. Although 
observers documented less time engaged on summary variables than teachers reported on the 
daily log, and teachers reported less time engaged on summary variables through the daily logs 
than they did on the course questionnaires, a consistent pattern of rankings was evident. 
Integrated and vocational classes typically received higher ranks and academic and special 
education academic classes typically received lower ranks. 

Failure to discriminate. Inconsistent findings of significant differences across the 
instruments or lack of significant differences on any instrument (i.e., coaching instruction 
strategy) may be attributable to several factors. First, none of the instruments could be shown to 
be exceptionally reliable in terms of interobserver agreement. Research staff demonstrated 
moderate to high agreement with each other, but teachers and research staff had lower levels of 
agreement. Therefore, inconsistent results across instruments may be due to the fact that certain 
instruments were less reliable indicators of differences among class types. Also, there was 
considerable variability within class types. 

Second, the phenomena of interest may not have been present or observable. Based on 
the observer logs, it appears that teachers and students in integrated curricula classes spent more 
time on the summary variables. However, the variables may not have been implemented to such 
a degree as to consistently and significantly distinguish integrated curricula classes from other 
types of classes. Also, important elements of integrated curricula may have been less observable 
than teachers intended; based on (a) anecdotal comments suggesting teachers believed certain 
features were implicit within activities, and (b) low levels of agreement between teachers and 
research staff. 



O 

ERIC 



69 



94 



Finally, it may be that the construct of integrated curricula as represented in the summary 
variables is limited or skewed. For instance, in special education classes, integration of academic 
concepts occasionally occurred with personal life skills activities rather than vocational 
activities. Personal life skills contexts may or may not be equally useful for demonstrating 
“relevance” of academic skills or creating opportunities for applied student activities, two of the 
summary variables investigated. Also, observers and teachers anecdotally reported having doubts 
about documenting an activity as integrated - despite obvious academic and vocational blending 
- if the academic expectations tended to be less than what they considered college preparatory 
level work. Embedding the idea of college preparatory level academic work in the definition of 
the content blend categories may have affected whether or not teachers and research assistants 
coded activities as “reinforced” or “fused” content blend. 

Implications for Practice and Policy 

This study has two important implications for classroom practice and education policy. 
First, findings suggest that students may experience different types of curriculum and instruction 
in integrated curriculum classes than in other more traditional class types. Second, valid 
measures of integrated curricula have several important uses for program evaluation. 

Students' Experiences in Integrated Curricula Classes 

In this study, integrated academic/occupational classes did in practice provide curriculum 
and instruction that were qualitatively different from traditional or special education academic 
classes. A clear difference was seen in the amount of time students were engaged in hands-on 
activities and were exposed to both academic and vocational content. Students in integrated 
curricula classes, according to observers, spent on the average half the classtime in hands-on 
activities. Almost a third of the classtime was devoted to activities that reinforced or fused 
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academic and vocational content. These applied and blended activities were rarely practiced in 
the traditional or special education academic classes. Integrated curricula classes also were more 
likely to enroll students with disabilities than the traditional academic classes. These findings 
suggest that students, including those with disabilities, who prefer an applied learning approach 
to academics may find a better match in classrooms that provide integrated 
academic/occupational curricula. On the other hand, special education students who spend a 
portion of their day in self-contained special education classrooms as well as in regular 
education integrated curricula classes may experience very different types of curriculum and 
instruction as they move from class to class. It may be important for teachers to recognize these 
differences and determine whether students who are mainstreamed require preparation or 
support to meet the different activity demands. 

It is interesting to note that applied activities and blended curricula also were found more 
often in traditional vocational classes than in traditional or special education academic classes. 

In this study, the vocational classes also were more likely to enroll students with disabilities. 
Although the finding that students often engaged in applied activities in vocational classes may 
not be surprising, the extent to which vocational class teachers also provided activities that 
reinforced or fused academic content may be less expected. This finding suggests the feasibility 
of policies aimed at upgrading the academic content of vocational classes while preserving their 
applied focus. It is possible that some vocational classes provide avenues to academic material 
in the same manner as integrated curricula classes. 

It has been proposed that a potential benefit of integrated curricula classes is that 
students would not have to choose between academic and applied occupational courses as is 
often the case in current high school settings (Benz & Kochhar, 1996; Phelps, 1992). This study 
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provides some preliminary support for this belief. However, this study can not address the 
question of whether the academic content was comparable across class types. Nor can it answer 
the policy question of whether the academic content was functional for all students, including 
those with disabilities and those considered academically talented. It may only be concluded that 
students in this study were more likely to experience both academic and vocational content in 
integrated and vocational classes. 

Program Evaluation 

Valid measures of integrated curricula have two important uses for program evaluation. 
First, the indicators may be used to monitor implementation of integrated curricula for purposes 
of program development in schools that adopt an integrated curricula approach. Second, the 
measures may be used to assess the relation between implementation of integrated curricula 
components in the classroom and student outcomes such as achievement in content areas, school 
engagement and graduation, and postschool success in employment and postsecondary 
education. As part of a system of school-level indicators these types of measures are a way. for 
policymakers and practitioners to purposefully shape improvement efforts at multiple system 
levels (Levesque, Bradby, & Rossi, 1996; National Study of School Evaluation, 1993). 

Monitoring Program Implementation . One use of the measures and instruments in this 
study is to support schools’ self-assessments and dialogues with parents and other stakeholders 
regarding the goals and methods used in school programs. This type of information would fit 
into a model-guided method of program implementation monitoring (Brekke, 1987). In this type 
of monitoring, data are collected that reflect the underlying purposes or theories of program 
operation. These data are then used systematically by school personnel and stakeholders to 
inform or refine existing programs. 



Teachers generally supported the belief that differences between class types found 
through the daily log represented important differences, suggesting some degree of face validity 
for the measures. Further, teachers expressed support for the idea that collecting classroom 
information about the measures would be useful for a variety of informational and professional 
purposes, such as monitoring how they use classtime to inform their own professional 
development. 

Any of the instruments would be feasible to use in a school self-assessment. Generally, 
the teachers did not view any of the reporting methods as overly intrusive in terms of time 
required nor disruptive to class activities. Daily logs offer advantages in terms of providing the 
most information and generated positive teacher response related to reflection on practice and 
professional development. However, it is possible that if additional training requirements were 
imposed to increase reliability or if a longer data collection period was required, the teachers 
would have been less positive about completing the logs. (It should be noted, however, that two 
teachers did not complete the study and were not included in the final sample because they were 
unable to regularly complete the daily logs.) The course questionnaire contained the most 
inflated estimates of time engaged on the summary variables, but has advantages of being easy 
to administer and requiring minimal training time. The observer logs were most reliable and 
relatively few observations were needed to characterize class types. However, extended training 
was required to maintain reliability. 

The measures and instruments meet several substantive and technical criteria for 
classroom practice indicators (Blank, 1993; Oakes, 1986; Porter, 1991). They focus on central 
features of schools related to curriculum and instruction; they are measurable, feasible to collect, 
readily understood and considered credible, and provide useful information to decision-makers 



(teachers, administrators and other policymakers). Although these measures alone are not 
sufficient for comprehensive monitoring (i.e., they do not address organizational or school 
context variables), they may serve as a valuable component of implementation evaluations 
related to integrated curricula reforms. 

Assessing Outcomes 

Most evaluation studies of programs involving integrated curricula have typically ignored 
levels of implementation in the classroom and have, therefore, failed to provide convincing 
evidence of the impact of integrated curricula. This lack of information makes it difficult for 
practitioners and policymakers in regular and special education to judge the value of an 
integrated curricula approach for students with and without disabilities. This study begins to 
address this problem by identifying measures that could be used to examine the link between 
what actually happens in integrated curricula classrooms and student outcomes, such as 
academic achievement. At the school or classroom level, these measures could be used to 
document degree of implementation. This information would provide greater insight into the 
results of evaluations purporting to show different outcomes for different class types. 

Given the overall moderate reliability of the measures and instruments, they should not 
be used as a means to identify cause-effect relationships or for accountability purposes (Porter, 
1993). However, the measures could inform policy by permitting exploration of the relative 
association between integrated curricula features and student and other program outcomes. For 
example, at the state or district level, outcomes of interest may relate to policy issues of equity 
and the value added to student outcomes by varying levels of resources delivered (Willms & 
Kerckhoff, 1995). The measures in this study, coupled with enrollment data, could be used to 
examine access by special populations to integrated curricula. Also, given the moderate 



correlation between rankings on instalments, course questionnaire results may serve as adequate 
indicators of some classroom variables in studies contrasting outcomes of integrated and 
traditional academic cuaicula. These comparisons would be strengthened by periodic 
observation of actual in-class practices and examination of relation between the course 
questionnaire and observation results. 

Limitations of Study 

This study must be considered an exploratory study. The sample was chosen to permit 
discrimination between class types within a comprehensive school setting. It might be argued 
that elimination of teachers from the final sample for the variety of reasons described in the 
sample section may have reduced the variability within class types and increased the possibility 
of significant findings. This would be of greatest concern in a study attempting to establish the 
relative influence of class type on student outcomes. However, for the purposes of this study, it 
was necessary to maintain “true types” in order to examine ability of the measures to 
discriminate among class types. In fact, there was considerable variability among individual 
classes within class types. 

The sample was not intended to be representative of other classes or schools. The results 
related to time engaged on the summary variables by different class types should not be 
considered representative of academic, vocational, integrated or special education academic 
classes in other schools. The measures were not designed to suggest that spending a specific 
amount of time on particular areas of curriculum and instruction would make a class 
“integrated.” Also, the measures did not identify significant differences between vocational and 
integrated classes. As discussed earlier, this may be a result of limited implementation in the 
sample schools, poor reliability, or that the five dimensions and their categories were not 



sufficiently defined to differentiate between the two class types. It is important to note that in 
situations where teachers might be held accountable for their performance on the measures, the 
data would be subject to further bias and would be considered less reliable than in this study. 

A further limitation of the study is that, although the measures were based on the 
available literature, the definitions used may not address areas of concern in other schools. For 
example, there was no attempt in this study to examine the depth or breadth of academic topics 
available in the classes. Also, the special education students in this study participated in self- 
contained academic classes or were mainstreamed in regular education classes. They were 
typically students with mild to moderate mental retardation or learning disabilities. If classes 
serving students with other types of disabilities (e.g., behavior disorders) or classes arranged in 
other ways (e g., special education self-contained vocational) had been included in the study, 
participants might have identified other dimensions of curriculum and instruction that should be 
considered as relating to integrated curricula. Similarly, another limitation of this study is that 
only school-based classrooms were examined. Classes that were based in the community or 
work settings may have important features that were not considered in relation to the measures 
tested in this study. 

Directions for Future Research on Integrated Curricula 

Based on these implications and limitations, four areas for future research on integrated 
curricula are examined below: (a) increasing validity and reliability, (b) program evaluation, 

(c) students with disabilities, and (d) personnel preparation. 

Increasing Validity and Reliability 

Several possibilities for future research are suggested by the identified limitations of this 
study. First, the validity of the measures should be further assessed in other schools, in both 
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school- and community-based classes, and with other types of students with disabilities. This 
type of research would permit investigation of the generalizability or transferability of the 
measures. Second, additional measures should be included that would help to extend and refine 
the definition of integrated curricula. These additional measures should systematically 
document: (a) blending of other functional contexts (e g., life skills) with academics, and 
(b) academic content and skill level of activities. Further, experts familiar with the literature and 
practices related to integrated curricula could be engaged in a consensus process to determine 
support for and relative weight given to these measures within a larger school-to-work 
conceptual framework. Also, it would be interesting to examine whether variability within class 
types would be reduced and results modified if the sample classes were chosen from within a 
limited set of academic content areas (e.g., mathematics). 

Pro gram Evaluation 

A fundamental issue is to determine which integrated curricula variables (if any) are 
critical to improving student outcomes. A related issue is to determine how much of any . 
particular feature or combination of features could be considered adequate implementation of 
integrated curricula. What degree of implementation is necessary to create differences in student 
outcomes? Is there a range of implementation that is effective? Because these questions have yet 
to be answered, critical components of integrated curricula found in the literature must still be 
considered descriptive rather than prescriptive (Chen, 1990). 

Another important question raised by teachers during this study was the degree to which 
teachers must make the connections between academic and vocational content areas explicit for 
students. This issue, in part, could be addressed through additional teacher or observer training 
regarding definitions of the categories used to code activities. However, questions regarding 



student perceptions should be asked also. One purpose of blending content is to make curricula 
more relevant to students. Student perceptions of relevance may relate to (a) explicit teacher 
statements about relevance, (b) the nature of materials used or the content and type of activity, 
(c) the student's prior knowledge or belief about relevance of activities, and (d) proximity of 
events for which the student believes the information will be useful. Future studies should 
explore each of these factors to determine their relative influence on student perceptions of 
relevance. 

There is also a need to more thoroughly examine the features of integrated curricula as 
they exist in the context of different schools and programs, especially given the multiple reforms 
influencing schools (Little, 1992). What other processes do schools implement in order to 
facilitate or complement integrated curricula? Do these other activities account for positive 
student outcomes as much as integrated curricula? Answering these types of questions would 
provide more information to policymakers who must decide the relative importance, costs and 
benefits of competing reform efforts. 

Finally, it may be useful to expand upon and revise the items within the course 
questionnaire. The purpose of further developing the course questionnaire would be to provide 
an efficient and more accurate representation of classroom activities. Such an instrument would 
be less time consuming and easier to administer. 

Students with Disabilities 

An issue identified by some teachers participating in this study concerns the emphasis on 
vocational content as a means to create functional contexts for academic material. Other 
contexts, such as life skills or high interest topics were not included in the measures' definitions. 
This issue raises the question of whether vocational, life skills, and high interest topics can be 



equally useful and meaningful for creating functional contexts for academic material. This 
concern is supported by the special education literature, which suggests that a high quality 
school-to-work transition curriculum can and should include life skills content (Benz & 

Kochhar, 1996). Addressing this issue requires further exploration of the purposes of blending 
content and opportunities to compare the effects of activities that place academics within 
different functional contexts. 

Another concern raised by teachers and observers related to the fact that in this study, the 
concept of college preparatory work was embedded in definitions of integrated curricula, 
specifically within the content blend dimension. In future studies, it may be useful to assess the 
academic level of activities separately. A related question concerns the role of academic content 
in curricula for students with severe cognitive impairments. Although school-to-work policy 
emphasizes that all students should have challenging academic content, how this concept 
translates into practice for some students requires further exploration and discussion. 

This study poses another problem regarding possible differences in vocational 
approaches for some students in regular versus special education. In the regular education 
literature, integrated curricula incorporates the idea of general vocational content and 
emphasizes that general vocational content should precede job-specific training. Does it benefit 
students with mild and severe disabilities as well as students without disabilities to experience a 
sequence of vocational courses and activities that progress from general to specific? The special 
education literature suggests that it is more functional to provide job-specific training to students 
with the most severe disabilities. For students with severe impairments, training within specific 
environments for specific skills followed by generalization to multiple environments or related 
skills is more effective (Cipani & Spooner, 1994; Wagner, 1991). An interesting finding in this 
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study is that the special education academic classes tended to rank higher for amount of time on 
general vocational content than other class types. Future studies should describe which types of 
students are exposed to what types and amounts of general vs. job specific vocational content. 

Finally, it may be important to further examine the types of supports delivered and 
needed to facilitate participation of students with disabilities in integrated classes. Previous 
studies of classrooms that include exceptional students suggest that regular education teachers 
often do not differentiate curriculum and instruction for those students (e.g., McIntosh et al., 
1993; Westberg et. al., 1993). Future research should examine whether the types of curriculum 
and instruction generally provided in integrated curricula classes meet the needs of students with 
specific disabilities. 

Personnel Preparation 

Another critical issue raised by this study concerns the training and experience of 
participating teachers in relation to class types. It is interesting to note that the integrated 
curricula teachers had education and professional backgrounds similar to the traditional 
academic teachers. However, their classes looked more like the vocational classes in terms of 
vocational content, content blend, instruction strategy, student activity and instruction grouping. 
This finding may have resulted from (a) the incidental and inservice training teachers received 
prior to implementing the courses, (b) teachers' attitudes or interests in promoting certain types 
of activities, or (c) the structure and components of the specific curriculum package used. This 
information has implications for the type of teacher preparation needed to successfully 
implement integrated classes. Further, these training, personnel, and curriculum structure 
variables need to be considered as possible secondary indicators relating to degree of 
implementation. 



Conclusion 



This study explored in a systematic manner important aspects of curriculum and 
instruction in secondary classrooms and how these relate to the construct of integrated 
academic/occupational curricula found in the literature. As a result of this study, (a) the 
construct of integrated curricula was shown to be observable and measurable, 

(b) features of current conceptual models of integrated curricula were shown to have validity in 
a variety of classrooms, (c) important limitations of the integrated curricula construct were 
identified, (d) the reliability of teacher reports was assessed, and (e) teachers’ assessments of the 
relevance of the resulting information were explored. 

Such implementation information is needed to determine if integrated curricula, a major 
component of current school-to-work transition initiatives, is, in fact, beneficial for all students. 
The School-to-Work Opportunities Act, the Perkins Act Amendments and Individuals with 
Disabilities Education Act Amendments of 1990 have created an opportunity to put in place a 
comprehensive school-to-work system that creates high academic expectations and promotes 
positive student outcomes for all students. To achieve this goal, more information is needed by 
policymakers and practitioners about classroom-level implementation of school-to-work 
activities and how these activities address the individual educational needs of students with 
disabilities. This study addressed this need in two important ways: (a) completion of a 
methodological component of indicator development that is generally not possible for school 
practitioners and administrators to undertake, and (b) demonstration of the feasibility of using 
observable classroom practice measures to describe integration of academic and occupational 
curricula. 
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APPENDIX A 



Coding Definitions 




[EXCERPTED FROM TEACHER AND OBSERVER MANUALS] 



Observation Categories: Instructions and Examples 

(Question 1, column 3) VOCATIONAL CONTENT 

Instructions : Circle the code that best indicates the function of any vocational subject 
matter addressed during the activity. The vocational content may be secondary to other content 
covered at the same time. However, the vocational function of the content must be explicitly 
referenced by the teacher or students or apparent from the materials given to the students. 

Examples: Examples are given after a brief definition of each code. 

NONE: Occupational information is not presented or referenced during this activity. 

*Students are completing an academic assignment, such as reading, reciting geography 
facts, or solving math problems. The teacher does not discuss how these activities relate 
to vocational options for the students, nor do the materials used by the students 
specifically mention a relationship to vocational knowledge or skills. Students do not ask 
about or discuss the occupational relevance of the material. 

GENERAL CAREER AWARENESS: Students are learning about types of industries, job 
requirements, or exploring personal career interests. The function of the activity is for students 
to have a broader information base rather than to learn a particular vocational skill. 

*Students are writing a report about the educational requirements and typical duties for 
three different jobs of interest to the student. 

*A representative from a local hospital presents information about the variety of job 
opportunities within the healthcare industry. 

*The teacher and students discuss the impact of environmental regulations on local 
manufacturers. 

GENERAL WORK BEHAVIORS: Students learn about or practice social and personal 
behaviors that contribute to successful participation in a work setting. 

*The students roleplay interactions with boss and coworkers. 

*The teacher acknowledges that students are demonstrating grooming appropriate for the 
workplace. 

*The students discuss conflict resolution in the workplace. 



GENERAL WORK SKILLS: The skill or knowledge being learned is useful in a variety of job 
types. 



•Students are practicing keyboarding. 

•Students are developing a business plan for a school-based student-run business. 
•The students and teacher discuss job-search strategies. 



*Students complete a variety of job applications and the teacher checks for neatness and 

accuracy. 

* Students practice different strategies for giving oral presentations. 

The teacher shows how to set up a computer database to track inventory. 

JOB-SPECIFIC SKILLS: The task is essential to or associated with a particular job within an 
industry and has limited applicability outside that job type. 

* Students are "prepping" food. 

"■Students are repairing a small motor. 

* Students are reading about welding procedures. 

The teacher demonstrates how to cut hair in a particular style. 



(Question 1, column 4) CONTENT BLEND 

Instructions: Circle the code that best indicates whether the activity's content is primarily 
academic, vocational, or some blend of the two. Consider overt statements made by the teacher 
and students about the purpose of the activity as well as the function of materials used. 

Examples: Examples are given after a brief definition of each code. 

ACADEMIC: The purpose of task is to acquire or improve reading, writing, math, science or 
social studies skills. Students may use vooationally-relevant skills to accomplish the task, but 
these are not discussed or evaluated by the teacher or students. 

"■Students are reading about the civil rights movement. 

* Students are learning the names of complex geometric figures. 

"■Students are reporting facts about Tennessee geography. 

The teacher is explaining how to write a term paper. 

ACADEMIC, REINFORCE OCCUPATIONAL: The primary instructional focus is the 
academic task and the use of vocational materials or skills is secondary. A vocational task may 
be completed or occupational information gained, but this is incidental to learning to exercise 
academic skills and knowledge. The vocational skill or knowledge may be considered basic or 
introductory. The teacher or students may suggest that the academic skills being learned are 
relevant to some occupational area. 

The teacher asks students to practice note-taking in preparation for writing a report on a 
career of interest to them. 

The students are observing a small chemical reaction, and the teacher describes how 
these chemicals are used by technicians in a particular industry. 

To explain an economic concept, the teacher uses illustrative information about a well- 
known business firm. 
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FUSED ACADEMIC/OCCUPATIONAL: The student is called upon to employ both academic 
and vocational skills or knowledge. The purpose of the task is equally focused on improving or 
acquiring academic and occupational knowledge or skills. The academic skills involved are 
typically more than basic (e g. academic concepts from college preparatory classes or 
representative of "higher order thinking"). 

’"Students must plan and prepare a meal for 50 people, including developing a work 
schedule, adapting recipes designed for smaller numbers, and completing a purchase 
order. 

"The teacher introduces an algebra skill by discussing the use of Ohm's law in the 
electronics field. 

* Students prepare a technical manual that could be used to train new workers. 

*The teacher explains the components of a successful business plan and how to present 
the plan to potential funders. 

•Students use biology concepts to investigate plant or livestock production on an existing 
farm. 

•Students report on working conditions in factories during the early part of the century as 
described in literature of the time. 

OCCUPATIONAL, REINFORCE ACADEMIC: The primary focus is the vocational task and 
the use of academic materials or skills is secondary. An academic task is completed or 
academic information is gained, but it is incidental to learning occupational skills or knowledge. 
The academic skills involved may be basic or remedial. The teacher may suggest that the 
vocational skills being learned make use of certain academic or theoretical concepts, or that 
certain academic skills have "real-life" importance. 

•Students are working with hydraulic lifts and the teacher mentions the physical 
principles involved. 

•Students are developing a resume, and the teacher reminds the students that they will be 
graded on spelling, clarity and completeness. 

*A health care professional describes the educational requirements for different types of 
nursing degrees, and emphasizes the importance of taking college preparatory science 
classes. 

•Students create a chart showing the major steps of a job task. 

OCCUPATIONAL ONLY: The purpose of the task is to acquire or improve occupational skills 
and knowledge in areas such as agriculture, business, consumer or occupational home 
economics, industrial arts, marketing, occupational health, technology, or trades and industry. 
Students may use academic skills to accomplish the task, but these are not discussed or 
evaluated by the teacher or students. 

•Students are practicing completing job applications. 

•The teacher describes the skill standards expected for a particular industry. 

* Students read about sustainable agriculture. 

•Students test an electronic circuit board. 
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Other Issues : "Basic" academics include demonstrating routine reading or computation 
skills and the use ot fundamental facts or rules to complete math, reading, science, or social 
studies applications below the college preparatory level. 



(Question 1, column 5) INSTRUCTION STRATEGY 

Instructions: Circle the code that best indicates how the teacher interacted with students 
during the activity. 

Examples : Examples are given after a brief definition of each code. 

MONITOR: The teacher has minimal interaction with students. The teacher's primary objective 
is supervision or assessment of tasks and behavior. 

The teacher sits at the desk and grades papers while students complete a test. 

"“While students work on a small group assignment, the teacher gives directives, such as 
"do this" or "don't do that" or "just do your best". 

The teacher has two groups of students completing alternate activities; he or she 
monitors one group while interacting with the other. 

LECTURE/REVIEW: The teacher gives out information, or the teacher checks student 
awareness of the "right answer" or specific facts. 

*The teacher stands at the front of the class and tells the students about the kinds of 
reference books available in the library. 

The teacher states that the class will review information for a test and then asks 
questions that require a factual or true/false answer. 

The teacher shows a video to give another perspective on a particular topic. 

LEAD DISCUSSION: The teacher facilitates students' verbal exploration of an issue or 
challenges students to explain concepts. The teacher expects students to do more than give the 
"right answer" or brief factual responses. During discussion, the teacher encourages students to 
explore a question posed by the teacher or other students, and other students may be expected to 
participate in the exchange. 

The teacher presents a problem and then asks "how" or "why" questions as several 
students offer answers. 

The teacher moderates a student debate regarding the merits of a vocational education. 
"■Students present solutions to a homework problem and the teacher asks them to explain 
how they arrived at the answer. 

MODEL/DEMONSTRATE: The teacher (or student under teacher's direction) completes an 
activity in front of other students and points out the steps involved in completing the activity 
properly. 
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*The teacher uses a computerized career guidance information system while students 
watch. At the same time, the teacher verbalizes the steps the students must use to operate 
the system andJocate the desired information. 

*The teacher circulates among students who are making models of geometric figures. If a 
student is having difficulty constructing the model the teacher shows each student how to 
use the materials. 

*The teacher shows the class how to use different types of carjacks and lifts, and then 
has several students demonstrate proper use of the equipment while other students watch. 

COACH: Coaching is an activity that may involve lecturing, modeling, leading discussion, and 
questioning. The teacher changes modes frequently to adapt to individual students' activities. If 
the teacher is coaching, the student usually is producing a written or material product and the 
teacher is posing questions or suggesting strategies that will guide the student toward completion 
of the product or problem solution. The teacher is interested in guiding but not completely 
directing the process. Students may be expected to work without immediate supervision for part 
of the activity. 

The teacher circulates among students, assisting them to locate the type of reference 
book that would be most useful for them, asking them questions about their research 
topic, and discussing possible strategies for presenting information in the report. 

The teacher checks on students' progress on completing a job application; the student 
isn't sure who to list as a reference and the teacher asks the student to think about who 
knows the most about the student's previous work experiences; the student names his 
own parent and a neighbor; the teacher asks the student to think about which of those two 
people a potential employer would be most interested in talking to and why. 

Two students tell the teacher that they cannot get an alarm to activate. The teacher asks 
several questions about the alarm circuit, while the students show what they have tried. 
The teacher reviews the operating logic of the alarm mechanism and suggests the 
students think about simpler ways to activate the alarm. 

Other Issues: ALTERNATE TEACHER: If another instructor leads the activity, consider the 
alternate person's interactions when coding instruction activity. 

If the regular teacher and the alternate teacher are doing similar activities with smaller 
groups of the class, code this as one activity. 

If the regular teacher and the alternate teacher are doing different activities with students, 
you may code this as two activities 21 as one activity and one alternate content. This choice will 
depend on the size of the groups involved, the purpose of the alternate content, and whether all 
students eventually come into contact with the same material. 



Instructions: Circle the code that best indicates what the students did during this activity. 
Consider actual student behaviors and teacher instructions regarding expected behavior. 



(Question 1, column 6) STUDENT ACTIVITY 
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Examples : Examples are given after a brief definition of each code. 

LISTEN/TAKE NOTES/TEST: Students initiate few interactions with the teacher or other 
students. Students are expected to be quiet or attend to the teacher. 

♦Students take notes while listening to a presentation by the teacher. 

* Students take a written test. 

DRILLED RESPONSE: Students complete a series of similar exercises or problems that are 
primarily abstract, symbolic, or factual. The teacher may ask a series of questions and students 
are expected to provide brief verbal responses - the "right" answer - or to repeat information 
recently given to them. There is usually no follow-up question to a student response unless a 
"wrong" answer is given. 

* Students complete a worksheet of math problems. 

* Students review previously completed worksheets with the teacher. 

♦Students give an answer in response to a teacher's question, and the teacher says "that's 
right" or "no" and then proceeds to the next question or to another student. 

♦Students recite facts from memory. 

♦Students read the text aloud. 

DISCUSSION/DISCOVERY: Students give longer responses and explain concepts or issues. 
The students present their own opinions or share new information. Students ask questions of 
other students and respond to other students' ideas. The teacher may probe or follow-up student 
responses to seek more information. 

♦Students describe their current work experiences and give examples of how they deal 
with coworker conflicts. 

♦Students compare solutions to a homework problem and negotiate a common solution. 
♦Students report on their research about the functions of the Supreme Court in the United 
States system of government, and pose related questions to other students. 

♦Students role play a job interview with another student, and then give each other 
performance feedback. 

LAB OR APPLIED WORK WITH ROUTINE PROBLEMS: (Applied) Students' work is 
concrete ("hands on") rather than abstract or symbolic. Students use "real-world" materials or 
deal with problems that would be encountered outside a high school setting. (Routine) The 
student is given a formula or model to use as a guide to complete the work. The problem- 
solving method is the same for a series of applied problems, although each problem may be 
slightly different from previous ones. 

♦The student completes each step in a science experiment and documents the outcome. 
♦The students use a variety of reference books from the library to locate information for 
a report; the students transfer the information to note cards in a format suggested by their 
teacher. 
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* After being shown how to operate a commercial dishwasher, the student practices the 
steps involved using the dirty dishes and equipment from a food preparation project 
completed earlier in the period. 

*The student follows the steps in a technical manual to practice using a computerized 
robot arm for grasping and lifting. 



LAB OR APPLIED WORK WITH NOVEL PROBLEMS: (Applied) Students' work is concrete 
("hands on") rather than abstract or symbolic. Students use "real-world" materials or deal with 
problems that would be encountered outside a high school setting. (Novel) Students will have to 
adapt a previously learned formula or model to solve a problem. Or, the students must decide on 
the best use of materials to accomplish the work. The problem solving method is different from 
models given previously, or the type of problem has not been encountered by the students 
before. 



’'‘The teacher indicates that the students must solve a new problem independently. 

’'‘The teacher and students enter new information into a computer simulation program on 
urban planning; whether the outcome will be "successful" is not predetermined by the 
teacher. 

*Students present the results of independent projects. 

’"Students select a problem of interest to them and document different strategies they 
might use to solve the problem. 

Other Issues: If students have completed work and are allowed to select an alternate 
activity (e.g., play computer games) and the selected activity is not directly related to the course 
content, then list the student activity as "alternate content" (Question 2) and code as many 
categories as possible. Then, write the words "free time" in the left margin next to the "alternate 
content" boxes. 

Worksheets could be classified as listen/take notes, drilled response, or applied work 
depending on the teacher's directions. For instance, if students use the worksheet to record 
information being presented at that time by the teacher, this might be "listen/take notes". If the 
students are filling in the blanks or completing a series of similar problems after receiving 
information from the teacher, then this might be "drilled response". If the students are 
completing word problems or writing short answers this might be considered "lab/applied work 
with routine problems" if the student must use "real world" materials or do "hands on" activities 
to get an answer. 



(Question 1, column 7) INSTRUCTION GROUPING 

Instructions : Circle the code that best indicates the grouping the teacher used to organize 
the activity and instruction. 

Examples : Examples are given after a brief definition of each code 

WHOLE CLASS: The teacher directs instruction to all students. All students receive the same 
information. Students may interact with a variety of other class members or only the teacher. 
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*The teacher is lecturing in front of the class. 

* All students are taking a test and the teacher is supervising. 
*The teacher .asks all the students to watch a demonstration. 



SMALL GROUPS: The teacher works with a few students at a time, and students interact with a 
small number of other students. 

*The teacher divides the class into four groups and assigns each group a different 
research task related to the topic of instruction. 

*The teacher instructs the students to get into groups of three and compare their solutions 
to a problem. 

PARTNERS: The teacher works with two students at a time and a student interacts with one 
other student only. 

*The teacher asks the students to work in teams of two to complete a lab experiment. 
*The teacher assigns a student who has successfully completed an assignment to assist 
another student who is still working. 

INDIVIDUAL: The teacher provides individual instruction to students and a student has 
minimal or no task-related interaction with other students. 

*Students are working on an independent research project, and the teacher circulates to 
individual students to respond to questions. 

*The teacher works with one student to correct problems on an earlier assignment, while 
other students watch a video. 

Other issues : Grouping that is simply based on the room's physical arrangement (e.g., 
students sit in small groups because the room has tables instead of desks) or grouping that- is for 
convenience (e.g., working in pairs because there are not enough computers to go around) 
should probably be coded as "whole class". However, if the teacher groups students to facilitate 
more individualized instruction or cooperative learning activities, this may be coded as "small 
groups" or "partners". 

Students working independently may not be receiving "individual" instruction. Only choose 
"individual" if the teacher provides individualized instruction to the majority of the students for 
the activity. 

If a large class is split in two (for instance, to permit the teacher and an aide to work on 
separate activities with the groups) then describe the instruction grouping as "whole class". 
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APPENDIX B 



Data Collection Instruments 




Administrative Course Description 



School ’ Teacher 

Course Title 

Semester Period 

Date (MM/DD/YY) 



1 . Check the category that best describes this course: 

academic 

vocational 

integrated academic/occupational 

other 



2. If available, attach a course description. 



3. Student Enrollment: Do not include a student in more than one of the subcategories under male and 
female. Some students will not be included in the subcategories. 



Total number of students enrolled for this semester 


Total number of males 


Total number of females 


Limited English Proficiency 


Limited English Proficiency 


Low Income 


Low Income 


Academically Talented 


Academically Talented 


Special Education 

Learning Disability 

Mild/Mod. Mental Ret 

Sev./Prof. Mental Ret. 

Behavior/Emot. 

Disorder 

Health or Sensory 

Impaired 


Special Education 

Learning Disability 

Mild/Mod. Mental Ret 

Sev./Prof. Mental Ret. 

Behavior/Emot. 

Disorder 

Health or Sensory 

Impaired 
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Course Questionnaire 



School Teacher Name_ 

Course I Semester 



Period Date (MM/DD/YY) 

1 . During this semester, approximately what percent of this course's class time will you devote to each 
of the following types of occupational content? 

None 

General Career Awareness 

General Work Behaviors 

Acquisition or Application of General Work Skills 

Acquisition or Application of Specific Job Skills 

2. During this semester, approximately what percent of this course's class time will you devote to each 
of the following types of content? 

Academic content only 

Academic content with some reinforcement of occupational learning 

Fusion of academic and occupational learning 

Occupational content with some reinforcement of academic learning 

Occupational content only 

Other 



3. Compared to other courses in the school, how would you rate the academic content of this course? 

1 2 3 4 5 

Less Rigorous Average More Rigorous 



4. Compared to other courses in the school, how would you rate the occupational or technical content of 
this course? 

1 2 3 4 5 

Less Rigorous Average More Rigorous 



5. Is this course part of a sequence of courses? yes no 

If "yes", name the required courses immediately preceding and following. 



6. Approximately what percent of students in this course are in the following paths? 

Academic/College Preparatory 

Vocational/Technical 

General 

Other 



7. During this semester, approximately what percent of this course's class time will you devote to each 
of the following types of instructional activities? 



Monitor — 

Lecture/Review 

Lead Discussion 

Model/Demonstrate 

Coach Individuals or Groups 

Other 



8. During this semester, approximately what percent of this course's class time will you expect students 
to engage in each of the following? 



Listen/Take Notes/Tests 

Drilled Response 

Discussion/Discovery Activities 

Lab or Applied Work with Routine Problems 

Lab or Applied Work with Novel Problems 

Other 



9. During this semester, approximately what percent of this course's class time will you instruct students: 

As a whole class 

In small groups 

As partners (2 students) 

As individuals 



10. The administration of this school identified this class as an "academic"* course. 
Do you agree with this description? yes no 

If you answered "no", briefly explain why you disagree. 



•Class type replaced with vocational, integrated academic/occupational or special 
education to match administrative designation 
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11. Teacher Educational Background: 



Degree(s) in 

in 

in 

12. Current Professional Certification: 



13. Professional Teaching Experience: 

Years at elementary level 

Years at secondary level 

Years at postsecondary level 

Total years teaching experience 

Years teaching at this school 

Years teaching this course 

Years teaching in subject area 

14. Education administration experience: 

Position: Years: 

Position: Years:. 

15. Other experience in education field: 



16. Professional/business experience outside education field: 



Field: 


Position: 


Years: 


Field: 


Position: 


Years: 


Field: 


Position: 


Years: 
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School — Teacher's Name 

Course ; Period Date 



1 . Describe the content taught/studied by the majority of the class. 



Brief Description 


Classtime 


Vocational 


Content 


Instruction 


Student 


Instruction 


:Topic and Activity 


Emphasis 


Content 


Blend 


Strategy 


Activity 


Grouping 




25% 


None 

General 


Academic 

Only 


Monitor 


Listen/ 

Take Notes/ 
Test 








Career 


Academic, 






Whole Class 




33% 


Awareness 


reinforce 


Lecture/ 


Drilled 






General 


occupational 


Review 


Response 


Small Groups 




50% 


Work 


Fused 


Lead 


Discussion/ 






66% 


Behaviors 

General 


academic/ 

occupational 


Discussion 


Discovery 


Partners 






Model/ 


Lab/Applled 


Individuals 






Work Skills 


Occupational 


Demonstrate 


Work- 






75% 




reinforce 




Routine 






Job-specific 


academic 


Coach 








100% 


Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 






33% 


Career 


Academic, 






Whole Class 




Awareness 


reinforce 


Lecture/ 


Drilled 






General 


occupational 


Review 


Response 


Small Groups 




50% 


Work 


Fused 


Lead 


Discussion/ 






66% 


Behaviors 

General 


academic/ 

occupational 


Discussion 


Discovery 


Partners 




Model/ 


Lab/Applied 


Individuals 






Work Skills 


Occupational 


Demonstrate 


Work- 










reinforce 




Routine 






75% 


Job-specific 


academic 


Coach 










Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 


Whole Class 






Career 


Academic, 








33% 


Awareness 


reinforce 


Lecture/ 


Drilled 


Small Groups 






General 


occupational 


Review 


Response 






50% 


Work 


Fused 


Lead 


Discussion/ 


Partners 




66% 


Behaviors 

General 


academic/ 

occupational 


Discussion 


Discovery 


Individuals 




Model/ 


Lab/Applied 








Work Skills 


Occupational 


Demonstrate 


Work- 










reinforce 




Routine 






75% 


Job-specific 


academic 


Coach 










Skills 


Occupational 

Only 




Lab/Applied 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 


Whole Class 






Career 


Academic, 










33% 


Awareness 


reinforce 


Lecture/ 


Drilled 


Small Groups 






General 


occupational 


Review 


Response 






50% 


Work 


Fused 


Lead 


Discussion/ 






66% 


Behaviors 


academic/ 

occupational 


Discussion 


Discovery 


Partners 




General 


Model/ 


Lab/Applled 


Individuals 






Work Skills 


Occupational 


Demonstrate 


Work- 








reinforce 




Routine 






75% 


Job-specific 


academic 


Coach 










Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 





TOTAL 100% 
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2. How many minutes of classtime were spent on activities not directly related to learning this course's 
content? < 5 minutes > 5 minutes* 

*lf more than 5 minutes-of classtime were spent on activities not directly related to the course content 
because of some unusual, incident, you may want to note this on the weekly log. 



3. Were any students given alternate content? 
Reason for alternate assignment 



no 



yes 



i this table to 



show i 



alternata content 



Brief Description:Topic, 


Classtime 


Vocational 


Content Blend 


Instruction 


Student 


Instruction 


Activity: #S(UDENTs 


Emphasis 


Content 




Strategy 


Activity 


Grouping 






None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 


Lecture/ 




Whole Class 




33% 


Awareness 


reinforce 


Review 


Drilled 








General 


occupational 


Lead 


Response 


Small Groups 




50% 


Work 


Fused 


Discussion 


Discussion/ 






66% 


Behaviors 


academic/ 

occupational 


Model/ 


Discovery 


Partners 




General 


Demonstrate 


Lab/Applled 
Work- Routine 


Individuals 






Work Skills 


Occupational 

reinforce 


Coach 








75% 


Job-specific 

Skills 


academic 




Lab/Applied 
Work - Novel 






100% 




Occupational 

Only 












None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 


Lecture/ 




Whole Class 




33% 


Awareness 


reinforce 


Review 


Drilled 








General 


occupational 


Lead 


Response 


Small Groups 




50% 


Work 


Fused 


Discussion 


Discussion/ 








Behaviors 


academic/ 

occupational 


Model/ 


Discovery 


Partners 




66% 


General 


Demonstrate 


Lab/Applled 


Individuals 






Work Skills 


Occupational 

reinforce 


Coach 


Work- Routine 






75% 


Job-specific 

Skills 


academic 




La b/Ap pl^c- 
Work - Novel 










Occupational 

Only 












None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 


Lecture/ 




Whole Class 




33% 


Awareness 


reinforce 

occupational 


Review 


Drilled 

Response 


Small Groups 






General 




Lead 




50% 


Work 


Fused 


Discussion 


Discussion/ 








Behaviors 


academic/ 

occupational 


Model/ 


Discovery 


Partners 




66% 


General 


Demonstrate 


Lab/Applied 
Work- Routine 


Individuals 






Work Skills 


Occupational 

reinforce 


Coach 








75% 


Job-specific 

Skills 


academic 




Lab/Applled 
Work - Novel 










Occupational 

Only 










25% 


None 


Academic 




Listen/ 










Only 


Monitor 


Take Notes/ 






33% 


General 

Career 


Academic, 


Lecture/ 


Test 


Whole Class 




50% 


Awareness 


reinforce 

occupational 


Review 


Drilled 

Response 


Smalt Groups 




General 




Lead 






Work 


Fused 


Discussion 


Discussion/ 








Behaviors 


academic/ 




Discovery 


Partners 




66% 


General 


occupational 


Model/ 

Demonstrate 


Lab/Applled 
Work- Routine 








Work Skills 


Occupational 




Individuals 




75% 


Job-specific 

Skills 


reinforce 

academic 


Coach 


Lab/Applied 
Work - Novel 










Occupational 

Only 
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Weekly Log 



School 


Teacher Name 


Course 


Date (MM/DD/YY} 


Period 


Week# 1 



1 Was it difficult to use the coding scheme for any of the activities you described on the daily logs this 
week? no yes 

If yes, identify the day and activity, then briefly describe the difficulty. 



2. Did the class engage in any special activities this week that were not adequately 
described in the daily logs? no yes 

If yes, briefly describe the activity, purpose and amount of time involved. 



3. Were there any unusual incidents that significantly altered your intended lessons this week? 
no yes 

If yes, briefly describe the incident(s) and day. 



4. Do you have any questions regarding the study? no yes 

If yes, please write the question. 
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Observer Log 

School Teacher’s Name 

Course " Period Date 



Ohsftrver Name; 



Brief Description :Topic 


Classtime 


Vocational 


Content 


Instruction 


Student 


Instruction 


and Activity 


Emphasis 


Content 


Blend 


Strategy 


Activity 


Grouping 






None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 






Whole Class 




33% 


Awareness 


reinforce 


Lecture/ 


Drilled 






General 

Work 


occupational 


Review 


Response 


Small Groups 




50% 


Fused 


Lead 


Discussion/ 






66% 


Behaviors 

General 


academic/ 

occupational 


Discussion 


Discovery 


Partners 




Model/ 


Lab/Applied 


Individuals 






Work Skills 


Occupational 


Demonstrate 


Work- 










reinforce 




Routine 






75% 


Job-specific 


academic 


Coach 








100% 


Skills 


Occupational 

Only 




Lab/Applied 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 






Whole Class 




33% 


Awareness 


reinforce 


Lecture/ 


Drilled 








General 

Work 


occupational 


Review 


Response 


Small Groups 




50% 


Fused 


Lead 


Discussion/ 






66% 


Behaviors 


academic/ 

occupational 


Discussion 


Discovery 


Partners 




General 




Model/ 


Lab/Applied 


Individuals 






Work Skills 


Occupational 


Demonstrate 


Work- 










reinforce 




Routine 






75% 


Job-specific 


academic 


Coach 










Skills 


Occupational 

Only 




Lab/ Applied 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 


Whole Class 






Career 


Academic, 










33% 


Awareness 


reinforce 


Lecture/ 


Drilled 


Small Groups 






General 


occupational 


Review 


Response 






50% 


Work 


Fused 


Lead 


Discussion/ 


Partners 




66% 


Behaviors 

General 


academic/ 

occupational 


Discussion 


Discovery 


Individuals 




Model/ 


Lab/Applied 








Work Skills 


Occupational 


Demonstrate 


Work- 










reinforce 




Routine 






75% 


Job-specific 


academic 


Coach 










Skills 


Occupational 

Only 




Lab/Applied 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 










33% 


Awareness 


reinforce 


Lecture/ 


Drilled 


Whole Class 








occupational 


Review 


Response 






50% 


General 

Work 


Fused 


Lead 


Discussion/ 


Small Groups 






Behaviors 


academic/ 

occupational 


Discussion 


Discovery 






66% 


General 


Occupational 


Model/ 


Lab/Applied 


Partners 






Work Skills 


Demonstrate 


Work- 

Routine 


Individuals 


reinforce 




75% 


Job-specific 


academic 


Coach 






Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 





TOTAL 100% 



99 



BEST COPY AVAILABLE 



2. Were any students given alternate content? no 



yes 



Brief Descriptlon:Topic, 


Classtime 


Vocational 


Content 


Instruction 


Student 


Instruction 


Activity 
# STUDENTS 


Emphasis 


Content 


Blend 


Strategy 


Activity 


Grouping 






None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 


Lecture/ 




Whole Class 




33% 


Awareness 


reinforce 

occupational 


Review 


Drilled 

Response 


Small 






General 




Lead 


Groups 




50% 


Work 


Fused 


Discussion 


Discussion/ 






Behaviors 


academic/ 

occupational 


Model/ 


Discovery 


Partners 




66% 


General 


Demonstrate 


Lab/Applied 


Individuals 






Work Skills 


Occupational 




Work- 








Job-specific 


reinforce 


Coach 


Routine 






75% 


academic 












Skills 


Occupational 




Lab/ Applied 
Work - Novel 






100% 




Only 












None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 


Lecture/ 




Whole Class 




33% 


Awareness 


reinforce 

occupational 


Review 


Drilled 

Response 


Small 






General 


Lead 


Groups 




50% 


Work 


Fused 


Discussion 


Discussion/ 






Behaviors 


academic/ 

occupational 


Model/ 


Discovery 


Partners 




66% 


General 


Demonstrate 


Lab/Applied 


Individuals 






Work Skills 


Occupational 




Work- 










reinforce 


Coach 


Routine 






75% 


Job-specific 


academic 












Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 








None 


Academic 




Listen/ 






25% 


General 


Only 


Monitor 


Take Notes/ 
Test 








Career 


Academic, 


Lecture/ 




Whole Class 




33% 


Awareness 


reinforce 

occupational 


Review 


Drilled 

Response 


Small 






General 


Lead 


Groups 




50% 


Work 


Fused 


Discussion 


Discussion/ 






Behaviors 


academic / 
occupational 


Model/ 


Discovery 


Partners 




66% 


General 




Demonstrate 


Lab/Applled 


Individuals 






Work Skills 


Occupational 




Work- 










reinforce 


Coach 


Routine 






75% 


Job-specific 


academic 












Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 






25% 


None 


Academic 




Listen/ 










Only 


Monitor 


Take Notes/ 






33% 


General 

Career 


Academic, 


Lecture/ 


Test 


Whole Class 






Awareness 


reinforce 

occupational 


Review 


Drilled 

Response 


Smalt 




50% 


General 




Lead 


Groups 






Work 


Fused 


Discussion 


Discussion/ 






Behaviors 


academic/ 




Discovery 


Partners 




66% 




occupational 


Model/ 






General 


Demonstrate 


Lab/Applled 








Work Skills 


Occupational 




Work- 


Individuals 




75% 


Job-specific 


reinforce 

academic 


Coach 


Routine 








Skills 


Occupational 

Only 




Lab/Applled 
Work - Novel 





100 



BEST COPY AVAILABLE 



Observer: 



Teacher: 



Date: 



3. Time class began: Ended: Total class time: mins. 

4. Number of students present for majority of classtime: 

Number of instructors and aides present for majority of classtime: 

5. Identify the setting in which most class activities took place: 

classroom 

other on-campus area 

6. How many minutes of classtime were spent on activities not directly related to learning this 

course's content? < 5 minutes > 5 minutes 

7. Describe any unusual incidents/time off content/interesting issues that occurred in the class: 
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Follow-up Survey 

Collecting Classroom Information: Course Questionnaire, Daily l.ogs. and Observations 

DIRECTIONS: Circle ONE answer for each question. Write comments on the back of this page. 

1. Completing the 3-page course questionnaire at the beginning of the project took: 

1 2 3 4 

10 mins. 11-20 21-30 more than 

or less mins. mins. 30 mins. 



2. 


Completing a single daily log (i.e., 


for one day's activities in one class) usually took: 




1 


2 


3 


4 


5 mins . 


6-10 


11-14 


more than 


or less 


mins . 


mins. 


15 mins . 


3. 


The format of the daily logs made documenting classroom activities: 




1 


2 


3 


4 


very 


fairly 


a little 


very 


easy 


easy 


difficult 


difficult 


4. 


How confident were you about choosing among the codes in each category on the daily log to describe 
classroom activities? 


1 


2 


3 


4 


very 


fairly 


a little 


very 


confident confident 


unsure 


unsure 



5. How disruptive was the presence of project observers in your classroom? 



1 



2 



3 



not at all a little fairly 

disruptive disruptive disruptive 



4 

very 

disruptive 



6. Having observers in the classroom influenced the way I documented and coded activities on the daily 
log: 



1 

not at all 



3 

some 



4 

a lot 



2 

a little 



[OVER] 



Comments about the course questionnaire: 



Comments about the daily logs: 



Comments about observers or classroom observations: 



O 
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Reporting Classroom Information: Your Class and Overall Project Results 



DIRECTIONS. Review the enclosed summary information. 

Then, circle ONE answer for each question below 



1. How well do you think the summary information about YOUR class activities represents activities 
that occurred in your class during the three weeks of the project? 



1 



2 



3 



4 



very 

well 



fairly only a not at 

well little all 



Comments: 



How well do you think the summary information about YOUR class activities reflects the types of 
activities that occur in your class during the entire semester? 



1 



2 



3 



4 



very 

well 



fairly only a not at 

well little all 



Comments: 



3. According to the teachers' daily logs, integrated curricula and vocational classes were more likely to 
offer general types of vocational content (awareness, behaviors, skills) than academic or special 
education classes. (However, this difference was not statistically significant.) Do you think these 
results accurately represent an important difference in the types of vocational content in the different 
classes? 

1 2 3 

yes no not sure 

Comments : 



[OVER] 
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4 . 



According to the teachers' daily logs, integrated curricula and vocational classes were more likely to 
blend (reinforce or fuse) vocational and academic content than academic or special education classes. 
(Also, this difference was statistically significant.) Do you think these results accurately represent an 
important difference in the content blend of the different classes? 

1 2 3 

yes no not sure 

C omment ; 



5. According to the teachers' daily logs, there was no difference in the amount of time teachers used 
coaching as an instruction strategy. Do you think this accurately represents an important way that 
the classes are alike? 

1 2 3 

yes no not sure 

Com m en t s, 



6. According the teachers's daily logs, students in vocational classes spent significantly more time 

engaged in applied/lab activities than those in all other classes. Students in integrated curricula and 
special education classes also were more likely to engage in lab/applied activities than students in 
academic classes (although this difference was not statistically significant). Do you think these results 
accurately represent an important difference between types of classes? 

1 2 3 
yes no not sure 

Comments: 
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7. According to the teachers 1 daily logs, special education teachers were least likely to use small 

groups/partners for instructional activities. (This difference was statistically significant.) Teachers in 
integrated classes were most likely to group students this way (although this difference was not 
significant.) Do you think these results accurately represent an important difference between types of 
classes? 

1 2 3 

y es no not sure 

Comments: 



8. According to the teachers' daily logs, special education academic classes spent more classtime on 
content Qih&r than vocational or academic (e.g., personal lifeskills). Do you think these results 
accurately represent an important difference between types of classes? 

1 2 3 

y es no not sure 

Comments: 

9. Having this kind of summary information about classroom activities might be useful for: (CHECK 
ALL THAT APPLY) 

describing some aspects of how I use classtime to other teachers 

describing some aspects of how I use classtime to school administrators 

describing some curricular and instructional differences between types of classes in my school 

monitoring some aspects of how I use classtime to inform my personal professional development 

monitoring some aspects of how I use classtime to inform a school-wide assessment of classroom 

activities 

monitoring some aspects of how curricula and instruction are implemented in my school 

other 

none of the above 



[OVER] 



Other comments about the summary information for YOUR class: 



Other comments about the summary information for ALL classes: 
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Purpose of the Project 

This project is a- structured observational study. The purpose of the project is to test 
whether the project's classroom observation system generates valid descriptions of curricula and 
instruction. There are several questions that we will try to answer by doing the study. For 
instance: 



1. Will the measures of curricula and instruction used in the observation system 
distinguish between three types of classes: 1) academic, 

2) vocational, and 3) those that blend both academic and vocational content? 

2. Are teacher-completed daily logs a useful way to collect information about what 
happens in classrooms? Are one-time course surveys or observers' logs better 
methods? 

3. Is the observation system useful for describing both regular and special education 
classroom activities? 

4. Is the information generated from the observation system meaningful and useful 
for teachers, administrators, or researchers? 



If the observation system is shown to be meaningful and useful, then teachers, 
administrators, and researchers could use it to: 

1 . describe important classroom activities in simple, observable terms, 

2. assess the types of curricula and instruction used in a variety of classes, and 

3. evaluate whether different types of curriculum and instruction are associated with 
student achievement and success. 
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Participating Teachers' Role 

Teachers are the primary data collectors in this project. The observation system we will 
use is designed for teachers to describe in a simple way some important aspects of high school 
classes. The information teachers record on the daily logs will be used in several ways to assess 
the observation system's usefulness. 

As a participating teacher, your major responsibilities are as follows. 

1. Meet with the project director individually (or with a small group) for 
approximately one hour to receive and review the project materials. Practice 
coding some classroom activities. 

2. Complete a brief questionnaire about the class(es) to be observed and yourself. 
This should take about 15 minutes and is done just once. 

3. Complete a daily log for each observed class for 3 weeks (15 consecutive school 
days). Each log can be completed in about 5 minutes. 

4. During the same 3 weeks, complete one weekly log each week. This form can be 
updated daily and at the end of each week. Each weekly log will take about 15 
minutes total to complete. 

5. Allow an observer to visit your class 4 or 5 times total during the 3 week study. 
The observer will use an observation log similar to the teachers' daily log to 
record information about curriculum and instruction. 

6. OPTIONAL: After the project has been completed, you will receive summary 
information about your observed class(es) as well as general project results. You 
also will be given a brief follow-up survey about the observation system data 
collection methods and results. 
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General Procedures 



During this project we will observe forty academic, vocational, and integrated 
academic/vocational classes serving students with and without disabilities in two local high 
schools involved in the High Schools that Work initiative. We will document normal classroom 
practices across five categories of curriculum and instruction: 

1 . vocational content, 

2. content blend, 

3. instruction strategy, 

4. student activity, and 

5. instruction grouping. 

University project staff an : 1 , school administrators will complete brief course descriptions 
and document student enrollment information (e.g., numbers of males/females, number of 
students receiving special education services). Research assistants (observers) will be trained to 
complete observation logs. The observers will visit each participating teacher's class(es) at least 
three times and as many as five times during the three week data collection period. 

Participating teachers will receive training from the project director and will complete 
one course questionnaire, fifteen daily logs, and three weekly logs over the three week data 
collection period. Following completion of the project, teachers will receive information about 
the project results. Teachers will be invited to respond to a follow-up survey about the project's 
data collection methods and results. 

A calendar showing the timeline for teacher activities follows this page. 




113 



Course Questionnaire 



The course questionnaire is to be completed independently by participating teachers after 
meeting with the project director to learn about the coding categories. You should be familiar 
with the coding categories before answering the questions on the course questionnaire. You may 
want to complete some practice daily logs before completing the questionnaire. However, you 
should return the completed course questionnaire to the project director before starting to 
formally record information on the fifteen daily logs. 

When considering your answers on the course questionnaire for items about the course's 
content and instruction, use the same definitions that were discussed during the meeting with the 
project director and that will be used on the daily logs. Examples of the categories and codes are 
given in this manual in the Daily Log section. 

NOTE: Questions 1, 2, 6 - 9 should each total 100%. 

The final page of the Course Questionnaire asks for information about your background 
and professional experience. When you are asked to list "years" involved in some professional 
experience on this form, give the total number of years rather than the actual dates (e.g., write 
"4", not "1975-1979"). 

Daily Log 

Complete each daily log as soon as possible after the observed class. You are more 
likely to accurately describe the class activities if you record your observations immediately 
after the class. It is important that you only record information about what actually happened 
during the class. Activities that were planned, but did not occur should qq! be listed on the daily 
log. 

Completed logs should be turned in on a daily basis to 
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The following sections will provide information on how to complete the daily logs. We 
have tried to give examples that will guide you in describing the activities in your classroom. 

We realize that classroom activities are complex and that the categories you have been given to 
describe those activities may seem overly broad or simple. Keep in mind that the daily logs are 
meant to record only some of the information about what happens in your class, and one of the 
project's purposes is to find out how well the observation system works. 

The examples given in this manual are not all inclusive, and you will need to use your 
professional judgement about the function of some activities in order to complete the logs! If 
you are unsure how to describe a particular activity, review the examples in the manual, and then 
try to document the activity on the daily log using the given categories and codes. 

After reviewing the examples, if you cannot decide between two codes within a category, 
you may circle your first choice, and then put a diagonal slash through your second choice. 

Then, make a note or ask a question about that activity on the weekly log. 

If you think an activity cannot be described using a particular category, first -/rite the 
brief activity/topic description in the first column, and assign a classtime emphasis code. Then 
code as many categories as you can. Put a large "X" through the category you are unable to 
code, and make a note about this on the weekly log. 

Observation Categories: Instructions and Examples 

(Question 1, first column) ACTIVITY/TOPIC 

Instructions : In a few words, describe each course-related activity that occurred in 
today's class and, in which, the majority of the class was engaged. In the space provided, note 
two things for each activity: 1) the general nature of the activity and 2) the main topic or content 
area. Put one activity in each space. Only list activities that are related to the course subject 
matter. If some students engaged in alternate course-related activities, refer to the section below 
on "Alternate Content". 
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Examples: One way to think about what to write is to decide what the title of the lesson 
plan for this activity might be. You do not need to be highly specific. For instance, do not give 
student performance objectives. However, do not use overly broad terms such as "math" or "job 
skills". Try to identify the main theme or content area within your subject area. Some examples 
are: 



"Conducting small-scale lab on ionic compounds" 

"Updating work-experience portfolios" 

"Reviewing Tennessee geography" 

"Watching a video about VICA membership" 

"Demonstrating proofs involving congruent angles" 

"Practicing clerical tasks (alphabetizing, filing, stamping)" 

Other Issues : Typically, a single activity takes at least 25% of the classtime, but this is 
not always the case. If an activity is very brief, you must consider whether it is actually a 
separate activity. For instance, if you give a brief review of previous class activities as an 
introduction to a new activity, you should probably consider the review as part of the new 
activity. However, if you spend 10 minutes reviewing a chapter in preparation for a test, and 
then switch to a lecture on another topic, you should list the review and the lecture as two 
separate activities. If you can not decide whether an activity should be listed separately, it is 
best to list the activity as if it were separate. 

If students are engaged in multiple or rotating activities (e.g., different work stations with 
slightly different content), but each activity would be coded in the same way, then give a general 
topic heading with each activity listed underneath, and circle the appropriate codes just once 
next to the general topic. 

If students in the class are engaged in more than 4 activities and each activity would be 
coded differently, select 4 activities that best represent the range of activities experienced by the 
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majority of the class. Then write the words "more than four" in the left margin of the Daily Log 
form. List any remaining activities as "Alternate Content" on the back of the form. 

If the class is interrupted for several minutes, or you must attend to matters other than 
course-related material, make a brief note about this on the weekly log and put a check mark on 
the daily log (Question 2) indicating that you spent more than 5 minutes on an activity that was 
not course-related. Do not list activities that are not course-related on the Daily Log. 

(Question 1 , column 2) CLASSTIME EMPHASIS 

Instructions : Circle the number that is closest to the percentage of classtime spent on this 
activity on this day. If you prefer, you may write in an amount other than those listed. However, 
total classtime for all activities listed for Question 1 should not exceed 100%. 

Examples: If the students worked on a single activity for the entire class period, you 
would circle "100%" under classtime emphasis. If students worked on two different activities in 
60 minutes - the first activity taking 20 minutes and the second activity taking 40 minutes - you 
would circle "33%" for the first activity and "66%" for the second activity. 

Other Issues : Usually, the total classtime emphasis for activities listed in Question 1 
should be 100% and should never exceed 100%. 

However, if you lose more than 25% of your classtime because of some disruption, you 
should indicate this in three ways. First, the total classtime emphasis will be less than 100%, 
reflecting the amount of time lost (e.g., total of 75% if you lost 25%). Second, indicate in 
response to Question 2 that more than 5 minutes were spent on activities unrelated to the course 
and, third, make a related notation on the weekly log. 

If you have a shorter disruption (for instance, one that lasts only 10 minutes), you can 
still calculate your total classtime based on portions of 100% and simply indicate that the 
disruption occurred in Question 2 and on the weekly log. 
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Time off content of less than 5 minutes will not be reflected in your total classtime 
emphasis or on the weekly log. 

Total classtime emphasis for "Alternate Content" (Question 3) may be less than 100%. 

(Question 2) TIME OFF CONTENT 

Instructions: Put an "X" next to the amount of time that is the best estimate of time spent 
during this class on other than course-related activities. If the class is off content for more than 
5 minutes, include a brief explanation on the weekly log. 

Examples: Behavior management problems that stop the teacher from interacting with 
other students. Dealing with interruptions from outside class. Processing several tardy slips. 
Listening to intercom announcements. Fire drills. Students have finished assignments and are 
waiting to leave. 

Other issues: If 1/2 or more of the class is disengaged (waiting to leave or between 
activities) for more than 5 minutes, consider this as time off content even if other students 
continue to work. Make a note about this on the weekly log. 

(Question 3) ALTERNATE CONTENT 

Instructions: Use this section to identify course-related classroom activities in which only 
some of your students took part. These may be activities that are not central to the day's lesson, 
but must be completed by a particular student to fulfill course requirements. Describe the topic 
and activity in the same manner as you would for activities listed in Question 1. 

Examples: A student takes a make-up test during class because of an earlier absence, 
while the other students work on the regular assignment. The test-taking is listed as an alternate 
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topic/activity in the table under Question 3 and the regular assignment is listed in the table under 
Question 1. 

Another example would be a class with students who regularly work on different 
activities. In this case, list the activity assigned to the larger group of students in the table for 
Question 1. List the activity being done by the smaller group of students as alternate content in 
the table for Question 3. 

Also, if a teacher holds individual teacher-student conferences regarding grades or 
assignments, include this under alternate content. 

Other Issues: The total classtime emphasis for "Alternate Content" may not add to 100% 
if a student or group of students worked for less than the entire class period on the alternate 
actitivity. It will total 100% only if a student or group of students was engaged in alternate 
content for the entire class period. 

Weekly Log 

notes as needed on the weekly log throughout the week and then finish the log at the end 
week. The purpose of the weekly log is to: 

1 . identify classroom activities that are not easily described with the observation 
system; 

2. identify features of classroom activities that may be important, but are not 
adequately described by the observation system; 

3 . identify any unusual incidents or periods of time off content that may have 
changed the class' typical content and instruction; and 
4. provide another way for teachers to ask questions about the project. 

Even if you answer each question with "no", submit the completed weekly log. 

Weekly logs should be given to 



Make 
of the 
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Purpose of the Project 



This project is a structured observational study. The purpose of the project is to test 
whether the project's classroom observation system generates valid descriptions of curricula and 
instruction. There are several questions that we will try to answer by doing the study. For 
instance: 

1. Will the measures of curricula and instruction used in the observation system 
distinguish between three types of classes: 1) academic, 

2) vocational, and 3) those that blend both academic and vocational content? 

2. Are teacher-completed daily logs a useful way to collect information about what 
happens in classrooms? Are one-time course surveys or observers' logs better 
methods? 

3. Is the observation system useful for describing both regular and special education 
classroom activities? 

4. Is the information generated from the observation system meaningful and useful 
for teachers, administrators, or researchers? 

If the observation system is shown to be meaningful and useful, then teachers, 
administrators, and researchers could use it to: 

1. describe important classroom activities in simple, observable terms, 

2. assess the types of curricula and instruction used in a variety of classes, and 

3. evaluate whether different types of curriculum and instruction are associated with 
student achievement and success. 
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Observers' Role 



The primary responsibility of the observers is to document classroom activities in a 
complete and consistent manner throughout the study. The observers' documentation will be 
used to assess the reliability of teachers' descriptions of curricula and instruction. The 
information recorded by teachers and observers will be used in several ways to assess the 
observation system’s usefulness. 

As an observer (research assistant), your duties include: 

1 . Meet with the project director and other research assistants to review the project’s 
purpose and materials. 

2. Become familiar with the coding system as presented in the observation manual 
and in training sessions with the project director. Practice coding classroom 
activities until you achieve interrater reliability of 90% minimum with the project 
director and other observers. 

3. Complete a set of observation notes and an observer log for each assigned class. 
Submit these documents on a timely and regular basis as requested by the project 
director. 

4. Meet weekly with the project director and other research assistants to review 
coding issues and to practice coding classroom activities in order to maintain high 
interrater reliability. 

5. Maintain accurate mileage and work logs. 
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General Procedures 



During this project we will observe forty academic, vocational, and integrated 
academic/vocational classes serving students with and without disabilities in two local high 
schools involved in the High Schools that Work initiative. We will document normal classroom 
practices across five categories of curriculum and instruction: 

1 . vocational content, 

2. content blend, 

3. instruction strategy, 

4. student activity, and 

5. instruction grouping. 

University project staff and school administrators will complete brief course descriptions 
and document student enrollment information (e.g., numbers of males/females, number of 
students receiving special education services). Research assistants (observers) will be trained to 
complete observation logs. The observers will visit each participating teacher's class(es) at least 
three times and as many as five times during the six week data collection period. 

Participating teachers will receive training from the project director and will complete 
one course questionnaire, fifteen daily logs, and three weekly logs over the three week data 
collection period. Following completion of the project, teachers will receive information about 
the project results. Teachers will be invited to respond to a follow-up survey about the project's 
data collection methods and results. 




125 



Observer Notes 

During each observation you are to make a condensed account of classroom activities as 
they occur. Use the preprinted "classroom observation notes" sheets to record your notes. These 
notes should consist of legible words, phrases, or sentences that reflect observable activity in the 
classroom. These notes do not need to be a smooth, complete narrative. However, the notes 
should provide enough information to help you select (and justify) a code for each category of 
interest. Important contextual details that you must record for each activity are listed on the 
observation notes sheets. 

Attach your classroom observation notes to the completed observation log. 

General Instructions for Observations: 

Be as unobtrusive as possible. 

Arrive at the classroom before the final class bell and stay in the classroom until the class 
dismissal bell. 

Upon arrival, briefly identify yourself to the teacher. Choose a seat that is on the side or 
in back of the classroom, but sit close enough that you can see the materials being used 
by the students. 

You should have few, if any, interactions with the teachers and students. Any 
interactions that do occur should be respectful and non-judgmental. 

If the teacher or students are moving around the room during the class, you may move 
around also as long as this does not disrupt the activity. Sometimes you will want to 
move around the room to see the different types of activities going on or to examine 
materials more closely. 
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You may look at materials given to students by the teacher or ask students about their 
assignment if this will help you identify the content. However, this should not interrupt 
the teacher's or students' activity. 

If some students leave the class to complete a course-related activity in another area of 
the school, you may follow them, make notes about their activity, and then return to the 
place where most of the students are working. 

If you cannot see an activity or cannot follow students who have left the room, make as 
many notes as you can about the activity and why you could not observe it. 

Observer Log 

Complete each observer log as soon as possible after the observed class. You are more 
likely to accurately describe the class activities if you code your observations immediately after 
the class. It is important that you only record information about what actually happened during 
the class. Activities that were planned, but did not occur should nol be listed on the log. 
Completed logs should be turned in on a daily basis to the project director. 

The following sections will provide information on how to code activities using the 
observer logs. Become familiar with the categories and coding examples given in this manual. 
Your most important job is to be thorough and consistent when coding. The classroom 
activities you will observe are complex, and the categories you will use to describe those 
activities may seem overly broad or simple. Keep in mind that the observer logs are meant to 
record information about only a few aspects of the classroom activities. You do not need to 
document everything that happens, but you must document the categories of interest carefully. 

The examples given in this manual are not all inclusive, and you will need to use your 
judgement about the function of some activities in order to complete the logs. If you are unsure 
how to describe a particular activity, review the examples in the manual, and then try to 
document the activity on the log using the given categories and codes. 
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After reviewing the examples, if you cannot decide between two codes within a category, 
circle your first choice, and then put a diagonal slash through your second choice. Then, 
document the nature of the coding problem on the observation note sheets. 

If you think an activity cannot be described using a particular category, first write the 
brief activity/topic description in the first column, and assign a classtime emphasis code. Then 
code as many categories as you can. Put a large "X" through the category you are unable to 
code, and make a note about this on the observation note sheets. 

Observation Categories: Instructions and Examples 

(Question 1, column 1) ACTIVITY/TOPIC 

Instructions : In a few words, describe each course-related activity that occurred in 
today's class and, in which, the majority of the class was engaged. In the space provided, note 
two things for each activity: 1) the general nature of the activity and 2) the main topic or content 
area. Put one activity in each space. Only list activities that are related to the course subject 
matter. If some students engaged in alternate course-related activities, refer to the section below 
on "Alternate Content". 

Examples: One way to think about what to write is to decide what the title of the lesson 
plan for this activity might be. You do not need to be highly specific. For instance, do not give 
student performance objectives. However, do not use overly broad terms such as "math" or "job 
skills". Try to identify the main theme or content area within your subject area. Some examples 
are: 



"Conducting small-scale lab on ionic compounds" 
"Updating work-experience portfolios" 
"Reviewing Tennessee geography" 

"Watching a video about VICA membership" 
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"Demonstrating proofs involving congruent angles" 

"Practicing clerical tasks (alphabetizing, filing, stamping)" 

Other Issues : Typically, a single activity takes at least 25% of the classtime, but this is 
not always the case. If an activity is very brief, you must consider whether it is actually a 
separate activity. For instance, if the teacher gave a brief review of previous class activities as 
an introduction to a new activity, you should probably consider the review as part of the new 
activity. However, if the teacher spent 10 minutes reviewing a chapter in preparation for a test, 
and then switched to a lecture on another topic, you should list the review and the lecture as two 
separate activities. If you can not decide whether an activity should be listed separately, it is 
best to list the activity as if it were separate. 

If students are engaged in multiple or rotating activities (e.g., different work stations with 
slightly different content), but each activity would be coded in the same way, then give a general 
topic heading with each activity listed underneath, and circle the appropriate codes just once 
next to the general topic. 

If students in the class are engaged in more than 4 activities and each activity would be 
coded differently, select 4 activities that best represent the range of activities experienced by the 
majority of the class. Then write the words "more than four" in the left margin of the log. List 
any remaining activities as "Alternate Content" on the back of the form. 

If the class is interrupted for several minutes, or the teacher must attend to matters other 
than course-related material, make a brief note and put a check mark on the log for Questions 6- 
7 indicating that the teacher spent more than 5 minutes on an activity that was not course- 
related. Do not list activities that are not course-related on the observer log. 
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(Question 1, column 2) CLASSTIME EMPHASIS 

Instructions : Circle the number that is closest to the percentage of classtime spent on this 
activity on this day. If you prefer, you may write in an amount other than those listed. However, 
total classtime for all activities listed for Question 1 should not exceed 100%. 

Examples: If the students worked on a single activity for the entire class period, you 
would circle "100%" under classtime emphasis. If students worked on two different activities in 
60 minutes - the first activity taking 20 minutes and the second activity taking 40 minutes - you 
would circle "33%" for the first activity and "66%" for the second activity. 

Other Issues : Usually, the total classtime emphasis for activities listed in Question 1 
should be 100% and should never exceed 100%. 

However, if the teacher lost more than 25% of classtime because of some disruption, you 
should indicate this in three ways. First, the total classtime emphasis will be less than 100%, 
reflecting the amount of time lost (e.g., total of 75% if time off content was 25%). Second, in 
response to Question 6 indicate that more than 5 minutes were spent on activities unrelated to 
the course. Third, make a related notation in response to Question 7. The incident or disruption 
will be documented on the observation note sheets as well. 

If a shorter disruption occurs (for instance, one that lasts 10 minutes), you can still 
calculate the total classtime based on portions of 100%. However, indicate that the disruption 
occurred in your responses to Questions 6-7. 

Time off content of less than 5 minutes will not be reflected in the total classtime 
emphasis or in Questions 6-7. 

Total classtime emphasis for "alternate content" (Question 2) may be less than 100%. 
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(Question 2) ALTERNATE CONTENT 



Instructions: Use this section to identify course-related classroom activities in which only 
some students took part. These may be activities that are not central to the day's lesson, but must 
be completed by a particular student to fulfill course requirements. Describe the topic and 
activity in the same manner as you would for activities listed in Question 1. 

Examples: A student takes a make-up test during class because of an earlier absence, 
while the other students work on the regular assignment. The test-taking is listed as an alternate 
topic/activity in the table under Question 2 and the regular assignment is listed in the table under 
Question 1. 

Another example would be a class with students who regularly work on different 
activities. In this case, list the activity assigned to the larger group of students in the table for 
Question 1. List the activity being done by the smaller group of students as alternate content in 
the table for Question 3. 

Also, if a teacher holds individual teacher-student conferences regarding grades or 
assignments, include this under alternate content. 

(Questions 3-5) CLASS LENGTH, PEOPLE, PLACE 

Complete Questions 3-5 based on information in your observation notes. 

(Question 6-7) TIME OFF CONTENT 

Instructions: Put an "X" next to the amount of time that is the best estimate of time spent 
during this class on other than course-related activities. If the class is off content for more than 
5 minutes, include a brief explanation under Question 7. Also use your response to Question 7 
to highlight any other interesting or unusual contextual information from your observer notes. 
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Examples: Behavior management problems that stop the teacher from interacting with 
other students. Dealing with interruptions from outside class. Processing several tardy slips. 
Listening to intercom announcements. Fire drills. Students have finished assignments and are 
waiting to leave. 

Other issues: If 1/2 or more of the class is disengaged (waiting to leave or between 
activities) for more than 5 minutes, consider this as time off content even if other students 
continue to work. 
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